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Glossary 


Academic 
standards 


Assessment for 
learning (AfL) 


Assessment 
literacy 


Backward design 


Collaborative 
assessment 


Fixed levels of achievement that students are 
expected to reach on their programmes of study. 


Assessment for learning recognises the power 
of assessment to drive student learning and the 
importance of fully integrating assessment into 
the curriculum through formative activities. 


Knowledge of the processes and standards of 
assessment, e.g. understanding assessment tasks, 
assessment criteria and academic standards ona 
programme of study. 


Beginning the design process by describing what 
students are expected to know, do and become 
by the end of the programme, and the standards 
they are expected to reach, and then working 
backward to design a syllabus that will support 
them to reach this goal. 


Involving students in the co-design of assessment 
activities (formative and summative), 
co-construction of assessment standards, marking 
and making judgements and giving feedback on 
peers’ assignments. 
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Credit 


Criterion- 
referenced 


Dialogic feedback 


Framework for 
Higher Education 
Qualifications 
(FHEQ) 


Inclusive 
assessment 


Module 


Norm-referenced 
assessment 


viii GLOSSARY 


A credit is a unit of notional learning hours. One 
FHEQ (see below) credit is equal to 10 notional 
learning hours; learning hours include lectures, 
seminars, tutorials, online discussions, formative 
and summative assessments. 


In criterion-referenced assessment, performance 
is judged against a standard, described in a set of 
criteria. It is possible for all learners to achieve the 
highest grades, if their work is excellent. Driving 
tests are criterion-referenced; it does not matter 
how many people pass as long as they reach the 
standard (cf norm-referenced). 


Feedback practices that provide opportunities for 
students to discuss, raise questions and critique 
the feedback they receive. 


Framework for Higher Education Qualifications 
(FHEQ) sets out the levels/standards expected for 
degree classifications in the UK. 


Assessment tasks and processes that 
accommodate all learners and their diverse needs. 
Well-designed inclusive assessment avoids the 
need for reasonable adjustments as the tasks and 
processes do not exclude any groups of learners 
(see Chapter 9). 


A unit of study. Modules can have different 
lengths, e.g. 15, 30 or 60 credit modules (see 
‘credit’ above) and can be combined to make a 
programme of study. 


Norm-referenced assessment involves comparing 
student performance and ranking students 
against each other and against a ‘norm’. In norm- 
referenced assessment, students are not judged 
against a fixed standard but against each other. 
Unlike criterion-referenced assessment (see 
above), it is impossible for all students to achieve 
the highest grades. 


Observed 
structured clinical 
exam (OSCE) 


Peer assessment 


Problem-based 
learning 


Programme of 
study 


Reliability 


Research-based 
learning 


Subject 
benchmark 
statements 


Observed structured clinical exams are used in 
professional programmes to assess competencies. 
In medicine, students move through ‘stations’ 
designed to test their competencies in interacting 
with patients and making clinical judgements. 
Each station is assessed by observers and 
performance is evaluated using a mark sheet. 


Student assessments (or oral presentations, videos 
etc.) are read (or viewed) and marked by their 
peers. Marking involves giving feedback and/or 
grades. Peer assessment can be used for formative 
or summative assessment (see Chapter 7). 


Problem-based learning is a type of inquiry-based 
learning (also known as enquiry-based learning) 
often used in professional studies. Typically, 
students work in groups to solve a professional 
problem; problems are suitably complex and have 
a range of possible solutions. 


A programme of study is a series of modules or 
course units at undergraduate or postgraduate 
level. Some programmes (typically professional 
programmes) are tightly constructed with core 
modules and very few options; other programmes 
can be loosely constructed with extensive module 
choice. 


Reliability entails markers making comparable 
judgements about the quality of assessed work 
(see Chapter 6). 


Research-based learning involves students and 
teachers in research or research-like activities as 
part of a programme of study. These activities 
may explore existing knowledge or build new 
knowledge (see Chapter 5). 


Subject benchmark statements form part of the 
quality code for higher education in the UK and 
describe threshold academic standards in a range 
of disciplines. 


GLOSSARY 


Validity 


Writing-to-learn 
(WTL) 


GLOSSARY 


Valid assessment tasks allow students to 
demonstrate that they have met the aims of the 
module/programme to the expected standard. 


Writing-to-learn tasks are typically short writing 
tasks that help students to learn disciplinary 
concepts, explore disciplinary knowledge and 
skills, and give practice in writing for a range of 
diverse audiences. 
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Chapter 1 
Introduction 


Welcome to this guide to assessment in higher education. 


Why a Guide on Assessment 


Assessment is a high-risk activity. Quality assurance procedures, the 
employability agenda and equality and diversity concerns all affect 
assessment practices. Governments, employers, funders, professional 
bodies and parents are all concerned stakeholders, and this makes 
assessment a high-risk, pressurised activity for both students and 
teachers. 

Teachers spend a lot of time on assessment and feedback, but often 
have very little professional development in how to design assessments 
and make reliable assessment judgements. Teachers grapple with issues 
such as unreliable marking and grade inflation. Inexperienced teachers 
can be asked to design courses and assessments with little pedagogical 
support. Greater access to higher education has resulted in a much more 
diverse student body and an increased need for inclusive assessment 
practices. Professional development is often done on the job, with 
variable amounts of guidance from more experienced colleagues. 

Assessment dominates students’ thinking in higher education and 
determines what they will focus on in their studies. Students entering 
higher education may face unfamiliar assessment processes and tasks. 
They may lack or have different understandings of academic standards 
and teacher expectations. They may misunderstand plagiarism guidance 
and encounter the perils of contract cheating and essay mills. Perhaps 


it is not surprising that student surveys report dissatisfaction with 
assessment and feedback (Office for Students 2019; Advance HE 2019a). 

However, there are existing solutions, or partial solutions, to these 
problems. Research on assessment in higher education and evidence 
from practice suggest many helpful strategies and activities that have 
been shown to be effective in creating less pressurised, more construct- 
ive assessment environments which benefit both learners and teachers. 
This guide to current research, theoretical and practical thinking on 
assessment in higher education aims to help you design more valid, 
inclusive assessments and develop strategies for ensuring more reliable 
assessment judgements. 


Academic Standards 


A key concept in the guide is the importance of academic standards, 
defined as ‘fixed reference levels of attainment’ (Sadler 2005, 193). Both 
students and teachers need to develop an understanding of academic 
standards for the particular modules and programmes they are involved 
in. Shared understanding of academic standards is key to ensuring 
assessment is reliable; reliable assessment promotes student satisfac- 
tion with their programme of study, as students better understand what 
they are aiming for and how they can better direct their own learning 
(Sadler 2010). Sadler (2010) argues that to move forward in their 
learning, students must conceptualise the gap between their current 
level, and the level they want to reach. They gain an understanding of 
how to bridge this gap through studying exemplars of their peers’ work 
and being guided to make judgements about the quality of these. In this 
way, students come to understand what makes good quality work in their 
discipline and can better peer- and self-assess. In Boud et al.’s (2018) 
terms, through this process students develop evaluative judgement and 
are better able to judge the quality of their own and others’ work and so 
direct their learning. 

This book proposes that curriculum and assessment design need 
to focus on helping students (and teachers) develop and share an under- 
standing of academic standards on their programmes of study. Practical 
ways of developing this understanding are suggested, e.g. through the 
use of exemplars (see Chapter 7 on Guided Marking), dialogic feedback 
(Chapter 8), and self-, peer- and collaborative assessment (Chapter 7). 


ASSESSMENT AND FEEDBACK IN HIGHER EDUCATION 


Linking Theory and Practice 


Like many professions, there is a gap in education between theoretical 
thinking and research work and educational practice. Ideas that are 
commonly discussed in the literature, such as dialogic feedback, collab- 
orative assessment and inclusive assessment, may not be reflected in 
educational practice in a programme of study. This guide offers a review 
of key ideas in the literature followed by illustrations which demonstrate 
how these ideas can be put into practice. Perspectives from theoretical 
and research literature are discussed, narratives of evidence-informed 
practice are presented in case studies and practical teaching ideas are 
suggested. The case studies and teaching ideas illustrate how you can 
draw on current perspectives in designing programmes of study and 
assessment. Each chapter ends with guidance on follow-up activities, 
such as further reading, links to assessment resources and ideas for 
small-scale investigations to help you better understand your educational 
context, your students and your assessment practices. 


Investigating Your Practice (and Why It is Important) 


This guide is aimed at academic teaching staff in higher education, both 
early career and experienced academics. I aim to give an overview of 
current issues in assessment, review theoretical and research literature, 
and show how practice can be informed by key perspectives so that 
theoretical ideas and research literature are linked with the practice 
of teaching and learning. This may seem an obvious statement, but 
often educational practice does not connect with research findings and 
theoretical thinking. 

There is a complex relationship between, on the one hand, research 
into education (which may be carried out by non-practitioners) and, 
on the other hand, the vast store of practical knowledge (Elbaz [1983] 
2019) which teachers acquire through their educational experience. 
Practical knowledge is rich, tacit and vague. Practical knowledge 
strongly influences practice; research on teacher beliefs and academic 
conceptions shows that previous learning and teaching experiences 
have a profound impact on a teachers’ practice (Elbaz 2019/1983; Borg 
2015; Samuelowicz and Bain 2001; Kember 1997). Teachers have deep 
and valuable knowledge about their discipline(s) and how best to learn 
and assess fundamental concepts. This valuable practical knowledge is 
often not connected, or not fully connected, with current educational 
research and theoretical thinking. Similarly, research findings from one 
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educational environment may not be applicable to another. Teachers 
need to make judgements about what will work in their context and with 
their diverse groups of learners. 

Teachers’ practical knowledge (see Chapter 2 for a detailed 
discussion) is a rich and powerful influence but is often not scrutinised. 
We know, or think we know what works; we might not know why it 
works. Small-scale action research and other methodologies are useful in 
helping academics explore practical knowledge, challenge assumptions 
and develop a reflective stance (Schön 2017). The follow-up activities in 
this guide suggest ways of exploring your own conceptions of teaching 
and learning, thinking about how these influence your teaching and 
assessment practices and what you might need to challenge. It is the 
premise of this book to try to bring together accounts of practice and 
theoretical and research work to offer academics a rich tapestry of ideas 
for supporting and assessing learning, to suggest ways of investigating 
your practice and of collecting evidence of effectiveness. 

Each learner is unique with a range of characteristics (e.g. age, 
disabilities, race, sexuality, access to wealth), prior learning experiences 
and diverse needs, and each teacher is equally unique. To investigate 
teaching and learning, it is important to choose a methodology that 
recognises and explores this uniqueness. Case study research is frequently 
used in education as it is suitable for small-scale research, is context 
specific and rich in detail (Simons 2014; Bassey 2003; Stake 1995). 
There are different traditions within case study research; Stake’s (1995) 
seminal work sets out a vision of case study research as an exploration 
of a unique phenomenon. He argues that generalisation is not the aim 
of case study research, instead it is the responsibility of the researcher to 
collect rich data on the case and come to know it well (Stake 1995, 8). 
Case studies may be designed to investigate a range of phenomena, 
for example, aspects of a degree programme, a group of learners, an 
individual learner and educational practices and beliefs of teachers. The 
type of data collected can range from interviews, observations, surveys, 
reflective journals (audio or written), analysis of students’ work and an 
enquiry into procedures and processes through the analysis of documen- 
tation (e.g. syllabi, module handbooks, assessment briefs and criteria, 
marking and moderation procedures). 

This approach to case study research fits well with an interpretivist 
research paradigm (Lincoln and Guba 1985). An interpretivist approach 
acknowledges the role of the researcher as constructor and interpreter 
of the research. As an investigator of your practice you will devise the 
study and interpret the data. The interpretation is yours, but through 
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working in collaboration with students and colleagues, you can collect 
and reflect on alternative perspectives. Co-operating with colleagues and 
working in a community of practice (see Chapter 2) is key to developing 
as a practitioner. Wenger-Trayner et al. (2015) explain the importance of 
belonging to a community of practice: 


You trust practitioners like your doctor to help you both for 
their experience and personal characteristics, and also because 
their actions reflect a competence defined by their community. 
Connection, engagement, status and a legitimacy in that 
community are all part of what makes someone a trustworthy 
practitioner. (2015, 14) 


Engagement with a professional community, with educational research, 
with investigation and reflection on one’s own knowledge and practice 
creates trustworthy teachers. 


How to Use this Guide 


This guide presents current issues in assessment, research and theoretical 
thinking in these areas and proposes principles of good practice. These 
principles are illustrated in cases studies and embodied in teaching ideas 
(see the tables below). However, what works in one context with one 
student cohort may not work with another. Teachers need to consider 
their own context and learner diversity and make judgements about how 
to adapt these principles to fit their context. A key principle in this book 
is that teachers and students work together to develop consensual under- 
standings of academic standards (this entails improving assessment 
literacy for both staff and students) and how programme design and 
learning activities can facilitate this. 

Chapter 2 gives an overview of key educational ideas and theories. 
These ideas are then explored further in each chapter. Chapter 2 covers: 


- how adults learn 

- academic standards 

- the role of exemplars 

- academic literacies 

- research-based learning 

- validity and reliability in assessment. 
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A key idea in this guide is that teachers and students require a shared 
understanding of academic standards. Without this understanding, 
teachers cannot make reliable judgements about the quality of students’ 
work, and students cannot judge their peers’ and their own work. 
Students who understand academic standards on their programme of 
study, can reliably self-assess and determine how to develop their work. 

Chapters 3 to 5 explore assessment design: Chapter 3 considers 
ways of ensuring validity in assessment through backward design of 
assessment, authentic assessment and preparing students for assessment. 
Chapter 4 focuses on assessment design across a programme; it discusses 
the problem of modularisation and the importance of connecting 
assessments across a programme. Authentic assessment is discussed in 
Chapter 3; this topic is then taken up and explored in detail in Chapter 5, 
which covers ways of reaching out to groups and organisations beyond 
higher education institutions and involving partners and students in 
research-based learning. 

While Chapters 3 to 5 focus on designing valid assessments, 
Chapters 6 to 8 problematise making reliable judgements. In Chapter 6 
the thorny issue of reliability in assessment is explored in conjunction 
with both teacher and student understanding of academic standards. 
Ways of ensuring more reliable assessment judgements are proposed. 
Chapter 7 focuses on students as assessors and explores the benefits and 
problems of involving students in assessment through peer and collabor- 
ative assessment. Finally, feedback, and helping students to understand, 
construct and use feedback, is discussed in Chapter 8. Throughout this 
guide, I stress the importance of ensuring that assessment and feedback 
practices are inclusive; Chapter 9 deals in detail with inclusive practice, 
arguing that good assessment practices are inclusive practices. I consider 
the benefits and challenges of offering choice in assessment. Chapter 10 
concludes the guide. 

If you want to dip into parts of the guide that are relevant for you, 
then you can use the assessment and feedback troubleshooting guide 
(Table 1.1) to find solutions to common assessment and feedback issues. 
If you are looking for practical ideas, Tables 1.2 and 1.3 will direct you 
to case studies and teaching ideas presented in the guide. Finally, if you 
want help with designing or reviewing aspects of assessment, Table 1.4 
will direct you to the design and review tools. 
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Table 1.1: Assessment and feedback troubleshooting guide 


This troubleshooting guide suggests solutions to some common 


assessment issues 


Issue 


Inconsistent feedback from a team of 


markers 
Differences in quantity and quality of 
feedback given 


Action 

Involve the team in peer review of 
feedback (Chapter 6). 

Use the feedback profiling tool 

to analyse feedback and improve 

the quality of feedback. Develop 
guidelines for the team of markers, 
standardising the format and amount 
of feedback. 

Use a feedback form to guide teachers’ 
comments. 


Inconsistent marking 

Lack of marker reliability 

Large team of inexperienced teaching 
assistants (TAs) 


Organise harmonisation/ 
pre-moderation sessions before any 
marking (Chapter 6). 

Get involved in calibration activities 
across a department or across a 
discipline (Chapter 6). 


Too many assessments 

No connections between assessments 
Students cannot use learning from one 
assessment in the next 


Carry out a Transforming the 
Experience of Students Through 
Assessment (TESTA). 

Focus on reducing assessments and 
ensuring links between assessments 
(Chapter 4). 


Late feedback or students do not 
receive feedback 


Use a tracking system for student 
submission of assignments and issue 
deadline reminders to markers. 

Flag any late return of feedback and 
take immediate action. 

Tell students why their feedback is late 
and when they will receive feedback. 


Students do not understand the 
assessment task and/or the standard 
they are required to reach and/or the 
feedback they receive 


Help students understand assessment 
and feedback by organising guided 
marking at the beginning of a module 
and introducing dialogic feedback 
(Chapter 7). 
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Students do not collect or open 
feedback and/or do not use it to 
improve their next assignment 


Ensure assessments are connected and 
show progressions so students need to 
take account of feedback (Chapter 4). 
Use a submission cover sheet that 

asks students to explain how they 
have used feedback from their last 
assignment to develop their submitted 
work. 

Remind students when grades and 
feedback have been returned and send 
a further reminder if they have not 
opened or collected their feedback 
within a week. 


Assessments do not test the 
knowledge, skills and competencies 
students are expected to learn 
Students do not perceive assessments 
as valid 


Use backward design (Chapter 4). 
Ensure that the syllabus links to the 
assessment and covers appropriate 
knowledge and skills. Provide 
opportunities for formative tasks that 
practise the knowledge and skills 
needed for the final assessment. 


Table 1.2: Illustrative case studies 


Learning from composing peer 
feedback 


Topic Level Discipline Chapter 
Research-based learning Undergraduate | Psychology 2 
Meet the Researcher and language 
sciences 
Research-based learning Undergraduate | Biology and 5 
Mystery specimen zoology 
Research-based learning Undergraduate | History of 5 
Creating a community of science 
researchers 
Calibration of academic Undergraduate | Accountancy 6 
standards (Australia) 
Collaborative assessment Postgraduate Education 7 
on an online masters 
programme 
Feedback and peer assessment | Undergraduate | Engineering 8 
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Table 1.3: Teaching ideas 


Topic Chapter 
Writing-to-learn (WtL) tasks 2 
Paraphrasing assessment criteria 7 
How to set up guided marking 7 
How to create a marker’s commentary 7 
Helping students understand assessment criteria (paraphrasing) |7 
Helping students understand feedback (paraphrasing) 8 
Table 1.4: Design and review tools and questionnaires 
Tool Chapter 
Module assessment design questionnaire 3 
Programme assessment design questionnaire 4 
Feedback profiling tool 8 


Follow-Up 


If you are interested in following up some of the ideas in this chapter, you 
might like to explore these resources. But you don’t have to! This isn’t a 
programme of study and this book is designed so that you can dip into 
sections that are relevant to your needs at any moment. I hope you will 
come back and explore further as your thinking develops. 


Reading 


Action research is suited to small-scale investigations of your practice. 
Jean McNiff’s (2013) guide explains the theory and practice of action 
research. 

McNiff, Jean. Action Research: Principles and Practice. 3rd ed. 
London: Routledge, 2013. 

Hutchings et al.’s (2011) guide to the scholarship of teaching and 
learning (SoIL) considers how teaching colleagues can work together 
with students to improve higher education. 

Hutchings, Pat, Mary Taylor Huber and Anthony Ciccone. The 
Scholarship of Teaching and Learning Reconsidered: Institutional 
Integration and Impact. San Francisco: Jossey-Bass, 2011. 
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Thinking 


Think about your disciplinary research practices. How do they differ 
from practices used in education, such as action research and case study 
research? 


Investigating Your Practice 


Investigate a learner. Meet informally with a learner (a current student 
on your programme, a past student, or a prospective student). Find out 
about their prior learning experiences, learning needs, concerns and 
ambitions. 

Discuss your findings with students and colleagues. What have 
you learned about teaching and learning on your programme and what 
changes would you advise? 

Investigate the learning environment. Shadow a student for a day. 
Find out how their day is organised, what pressures they are dealing with 
and what helps them study effectively. Are there aspects of the learning 
environment that hamper effective learning? What are they and how can 
they be improved? 

Investigate assessment documentation, e.g. assessment infor- 
mation in module handbooks. What do staff and students think of the 
information presented? Could the information be presented more 
clearly? Read more about designing assessment in Chapters 3 and 4. 
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Chapter 2 
Key Perspectives 


This chapter explores some key ideas in current thinking on assessment. 
I begin by considering how adults learn, since an understanding of 
learning guides how we design and assess programmes of study. 
Following this, I explore the nature of academic standards and why it is 
difficult for assessors to make reliable assessment judgements. An under- 
standing of academic standards is essential for designing assessment. 
The teacher needs to be clear about what standard of work they expect 
from students and students need to know what standard they need to 
reach. Academic standards guide the design of formative and summative 
assessment, facilitating student learning and achievement. 

Students benefit from being active participants in assessment, 
capable of self- and peer-assessing. To do this they need to be competent 
assessors with a good knowledge of academic standards. In order to 
achieve this, Tai et al. (2018, 5) argue that they need to develop evaluative 
judgement, which is defined as the ‘capability to make decisions about the 
quality of work of self and others’. Evaluative judgement is a prerequis- 
ite for successful peer and collaborative assessment, as discussed below. 
Developing evaluative judgement enables students to make judgements 
about their own and others’ work and to direct their own learning. 

In this guide, students are conceptualised as active learners 
making links between their higher education studies and the community 
beyond. There is a need to design assessment opportunities that allow 
students to make these links and to consider how research can benefit 
their communities. Research-based learning (RBL) is proposed as a way 
forward; there is a short introduction to RBL in this chapter with more 
consideration given to the approach in Chapter 5. In this chapter, how 
students learn to communicate their work and develop oral and written 
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communication skills is considered, drawing on the concept of academic 
literacies. This chapter concludes by considering two key concepts in 
assessment, namely, reliability and validity. 


How Adults Learn: the Concept of Communities of 
Practice 


How do students learn in higher education and how can they be supported 
to develop? To answer these questions, it is helpful to begin with some 
thinking around how professionals develop, including teachers’ profes- 
sional development. A dominant view in the literature is that profes- 
sionals learn in a ‘community of practice’. Since it was first proposed 
in Jean Lave and Etienne Wenger’s much cited book, Situated Learning 
(1991), this idea has become hugely influential in education and profes- 
sional learning programmes. They coined the term ‘situated learning’ to 
describe the experiences of apprentices learning through practice and 
participation in a community. Wenger (1999) argues that we are learning 
all the time; learning is not something that happens within the confines 
of educational institutions. He argues that, throughout our daily lives, 
we are constantly learning and that we do most of this learning through 
social participation in communities of practice. Wenger characterises 
social participation ‘as a process of learning and knowing’, identifying 
four components: 


1. Meaning: a way of talking about our (changing) ability — indi- 
vidually and collectively — to experience our life and the world 
as meaningful. 

2. Practice: a way of talking about the shared historical and social 
resources, frameworks, and perspectives that can sustain 
mutual engagement in action. 

3. Community: a way of talking about the social configurations in 
which our enterprises are defined as worth pursuing and our 
participation is recognised as competence. 

4. Identity: a way of talking about how learning changes who we 
are and creates personal histories of becoming in the context of 
our communities. (1999, 5) 


We belong to multiple communities of practice, e.g. families, work 


communities, disciplinary communities, and universities, and we learn 
through being active participants in these communities. The community 
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establishes practices and shares knowledge through participation in 
tasks. Individuals learn through engaging and contributing to community 
activities; knowledge is socially constructed within the community. 
Wenger argues that practices owned by a community change over time; 
he emphasises the ‘historical and social context’ of a community of 
practice (1999, 47). Acommunity of practice is not a hermetically sealed 
entity but part of a wider social context (Wenger 1999). 

Lave and Wenger developed the idea of a community of practice 
through studying apprenticeships; they were interested in how novice 
apprentices learn and develop through participation in a community 
and how they experience ‘identity transformation’ (1999, 11). Wenger 
argued that new participants are initiated into the community’s practice 
through ‘modified forms of participation that are structured to open the 
practice to non-members’ (1999, 100); Lave and Wenger (1991, 27) 
describe this process as ‘legitimate peripheral participation’. Legitimate 
peripheral participation entails engaging in introductory tasks that 
initiate the new member into the community of practice. Applying this 
idea to higher education, undergraduate students entering the university 
are novices who, through participation in the community’s activities, for 
example, research-like activities, gradually become members of a discip- 
linary community and begin to know and critique disciplinary practices. 
In online fora, inactive members who do not contribute to a discussion, 
but are reading posts and following the discussion, are called ‘lurkers’, 
though a more neutral term is vicarious learners. However, lurking in 
a community of practice is legitimate peripheral participation (Wenger 
et al. 2009, 9); lurkers are learning about the community, absorbing 
knowledge and practice, even if they do not feel ready to contribute to 
the discussion. 

Higher education teachers belong to multiple communities of 
practice, e.g. disciplinary groups, research groups, professional societies, 
research teams and programme teams. Their experience of learning 
within these teams influences their beliefs about teaching and learning 
in their discipline. Working together as a teaching team they develop 
community practices around teaching and assessment, so that the 
academic standards on their programmes become ‘socially constructed’ 
(Sadler 2014) within the teaching team. 

Introductory activities, such as Meet the Researcher (see Case 
study), bring students into the community of practice and facilitate 
learning about disciplinary thinking and research methodologies. 
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Case Study: Meet the Researcher - a Legitimate Peripheral 
Participation Activity 


Level — first year, first term, undergraduate 

Discipline — psychology and language sciences 

Cohort- 150 undergraduate students (home and international students) 
Learning approach — communities of practice and research-based 
learning 


The activity helps introduce students to the research communities in 
the faculty and promotes legitimate peripheral participation. Students 
entering the department engage in an activity to develop an understand- 
ing of departmental research. 

In induction week, the activity is explained to students and there is 
further guidance on the virtual learning environment (VLE) (Evans et al. 
2018). Students view short, professionally made videos of 42 researchers 
talking about their research. The researchers were asked to answer the 
following questions. 


What is your major research question? 
Why is it important? 
What have you found? 
(Evans et al. 2018, 304) 


Students are divided into groups of three to five and after viewing the 
videos, choose a researcher to interview. In order to formulate interview 
questions, they need to find out more information about the researcher’s 
work. The interview lasts from 30-60 minutes and students then prepare 
a short presentation on the researcher to present to their seminar group 
(seminar groups are composed of six to ten students). Students receive 
guidance on how to structure the presentation and use slides. After the 
seminar presentation, the personal tutor gives feedback. 

A follow-up survey showed that students valued the activity (n=47 
respondents); 90 per cent agreed with the statement: ‘I enjoyed meeting 
my allocated researcher.’ And 94 per cent agreed that the Meet the 
Researcher activity was an ‘overall good experience’. However, students 
complained about having to watch all 42 videos. 

The researchers who participated in the activity were very positive 
about the experience, e.g. one commented that students were well 
prepared and the videos of good quality. Personal tutors also gave positive 
feedback, commenting, for example, on how the activity motivated 
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students and helped them learn about presentation skills, the research 
culture in the department and interviewing skills. One student who 
took part in the activity, reflected at length on what he had learned and 
suggested that the interview should be about ‘the process of becoming a 
researcher’ (Evans et al. 2018, 309) as well as the research topic. 

Dwyer (2001) reported on a similar activity in geography. She 
saw the value of the activity in linking teaching and research and in 
introducing students to geography theory and philosophy. However, her 
study on the students involved in the activity suggested that they had 
variable experiences. She proposed facilitating a workshop before and 
after the activity to draw together key ideas. 

This sort of activity is a useful identifying activity (Levy and Petrulis 
2012; and see Chapter 5 below for more on identifying activities) that 
begins to induct students into ways of thinking within the discipline 
and enables them to enter a community of practice. The activity can be 
adapted so that it is more open, with students deciding how they will 
investigate and report on the researcher (suitable for postgraduate 
programmes), or more controlled with allocated researchers and set 
questions (more suitable for first year undergraduate programmes). The 
activity can be extended by involving researchers in talking about peer 
review comments on recently published work. The researchers share 
their responses to the feedback and describe how they used peer review 
comments to develop their work. This activity mirrors experiences 
that undergraduate and postgraduate students will have of receiving 
feedback on academic work, and hopefully provides them an insight 
into academic disciplinary practices. Both students and researchers are 
engaged in receiving and making sense of feedback; both are members 
of the same community of practice, though students may be peripheral 
participants. 


Further Reading 


Evans, Julie, Alex Standen, Alastair McClelland and Siir Saydam. 
‘Meet the Researcher: The Use of Interviews to Connect First-Year 
Undergraduate Students to Research Staff at UCL’. In Shaping Higher 
Education with Students: Ways to Connect Research and Teaching, edited 
by Vincent C.H. Tong, Alex Standen and Mina Sotiriou, 303-10. London: 
UCL Press, 2018. 

Dwyer, Claire. ‘Linking Research and Teaching: A Staff—-Student 
Interview Project’, Journal of Geography in Higher Education 25, no. 3 
(2001): 357-66. 
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Academic Standards 


Academic standards are the standards students are expected to achieve 
in their assessed work on a course. These standards are often described 
in written documentation such as subject benchmark statements (see 
Glossary) and assessment criteria. Markers make assessment judgements 
about the quality of student work, drawing on their understandings of 
academic standards (see Chapter 6 for a discussion of marker reliability). 
These judgements are based on the individual marker’s construction 
of the standard, which will be influenced by a range of factors, such as 
experiences as a learner, as a marker and disciplinary practices. The 
range of influences on the markers’ standards can create very different 
judgements on the quality of student work; documentation may be 
helpful, but written standards are open to interpretation. Documents and 
regulations do not create standards — practitioners do, through engaging 
in activity, such as calibration (Sadler 2014; Watty et al. 2014), creating 
socially constructed (Sadler 2014; Shay 2005) standards through 
discussion of exemplars, and a common interpretation of documented 
standards (see Chapter 6). 

A team of teachers, working together on assessment can be concep- 
tualised as a community of practice. The team may come from different 
disciplinary communities, each with their divergent epistemological 
understandings. With the rise in interdisciplinary programmes of study 
and new fields of study, e.g. biomedical engineering, academics and 
students are increasingly working in multi-disciplinary fields of study. 
Programme teams hail from a range of disciplines, with potentially 
different practices and epistemological understandings. In Lave and 
Wenger’s terms, individuals in those teams will belong to several 
communities of practice; for example, their initial disciplinary community 
of practice, their professional community of practice, and their inter- 
disciplinary team community of practice. These individuals develop a 
complex range of identities and values. Teams that work closely together 
and engage in calibration activities can create socially constructed 
academic standards and develop agreed assessment practices (e.g. 
around assessment design, academic standards, range of grades, good 
quality feedback). New markers joining the group are initiated, through 
legitimate peripheral participation into assessment practices and can 
develop an understanding of socially constructed standards and how to 
uphold them (see Chapter 6 for a fuller discussion). 
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In a team with a range of disciplines, the interplay between 
different communities of practice, and the overlap between different 
professions can be a source of conflict. A community of practice is 
socially constructed; practices are influenced by power dynamics within 
the group (hierarchies) and can change over a period of time as the 
membership of the community changes. Written rules and regulations 
cannot ‘fix’ the practices and standards in the group, as the written rules 
are always open to interpretation. Power dynamics in the team can mean 
that minority voices are silenced (Wenger-Trayner et al. 2015, 15-16). In 
a programme assessment team, the team would work together to create 
shared academic standards about the quality of student work. Through 
participation in activities such as marking, moderation and harmonisa- 
tion, they would agree academic standards. As the membership of the 
team changed over time, practices and standards would also shift as 
new members and external influences, such as government policies, 
influenced the community. This perspective implies that academic 
standards can be shared within a community but cannot be fixed over 
time as shifting membership in the group will create new interpretations 
of standards. 

Students are also participants in a disciplinary community of 
practice, though at the early stages of their study programmes this partic- 
ipation will be peripheral. As they move from the periphery, students 
need to develop an understanding of the community’s standards. This 
is often interpreted as explaining assessment criteria to students, but 
as I have argued above, assessment criteria do not describe standards. 
Students gain an understanding of academic standards through partici- 
pation in the assessment practices of the community; this entails 
involving students more in assessment processes; for example, as peer 
reviewers, peer assessors and in collaborative assessment. However, 
we know that students who have engaged in peer and collaborative 
assessment are often critical of their experiences (Wilson et al. 2015). 
The discussion below outlines some of the problems with peer and 
collaborative assessment, and possible solutions (see also Chapter 7). 

Research on marking in higher education provides evidence that 
teachers draw on a range of tacit knowledge in making a judgement 
about the standard of students’ work, and this practice of drawing 
on tacit knowledge results in variation in judgements (Sadler 2014; 
Bloxham et al. 2015). Regulations and criteria are designed to promote 
common standards, but they are limited in that the discourse used needs 
to be interpreted and can be interpreted differently by different markers 
and teams of markers. In the UK, the Quality Assurance Agency (QAA) 
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facilitates the writing of subject benchmark statements, which ‘describe 
the nature of study and the academic standards expected of graduates 
in specific subject areas. They show what graduates might reasonably be 
expected to know, do and understand at the end of their studies’ (QAA 
2019a). 


The Role of Exemplars 


Exemplars can help teaching teams to explore their understandings 
of criteria and, through discussion, create either consensus or a better 
understanding of differences in standards. Exemplars are used in pre- 
moderation (Bloxham et al. 2016) or harmonisation meetings, allowing 
the team to share their marking judgements and through discussion, 
explain what they value. These values can be more clearly illustrated 
through reference to exemplars, helping the team to come to a more 
consensual understanding of academic standards. For example, it may 
be difficult to describe criticality, but, by referring to exemplars, teachers 
can explain what they mean by ‘criticality and, importantly, what that 
looks like in students’ work. With a range of exemplars, different levels 
of criticality can be explored, e.g. exemplars can show both excellent and 
borderline attempts at criticality. 

Where national disciplinary standards are required, an approach 
piloted in Australia entails using calibration activities to construct 
national standards. Disciplinary teams from a range of institutions 
engage in a scaffolded discussion of exemplars, in order to agree 
the characteristics of excellent, good and borderline assignments at 
threshold and final year degree classifications (Watty et al. 2014, see 
Chapter 6 for a fuller discussion). It is not just teachers who need to 
develop assessment literacy and engage in calibration of standards; 
learners also need to have a clear understanding of the academic 
standards on their programme and what good quality work looks like. 
This aspect is often neglected in higher education; quality assurance 
processes often require programmes to describe academic standards 
through assessment briefs and assessment criteria and to ensure that 
students have this information. Teachers provide feedback to students, 
explaining the quality of their work. However, students find the profes- 
sional discourse in briefs, assessment criteria and feedback difficult to 
understand (Price et al. 2010; Ivanič et al. 2000). It also takes time for 
them to acquire assessment literacy and develop a full understanding 
of the disciplinary standards. Sharing and discussing exemplars can 


ASSESSMENT AND FEEDBACK IN HIGHER EDUCATION 


help develop a shared understanding of academic standards; sharing 
standards brings students into the disciplinary community of practice 
and prepares them for involvement in other assessment practices (see 
Chapter 7). 


Peer and Collaborative Assessment 


With the emphasis on assessment for learning, rather than assessment 
of learning, more attention has been given to involving students in 
assessment (Sambell et al. 2013). There are good reasons for this; the 
better students understand assessment standards and processes and 
develop assessment literacy around standards, the more likely they 
are to produce work which reflects their capabilities. Ways of involving 
students in assessment include collaborating with them to design 
the task and assessment criteria. As students develop competency in 
making assessment judgements, they can be involved in assessing 
and giving feedback to peers’ work. While peer assessment has been 
widely advocated and researched (Wilson et al. 2015), it is less widely 
implemented. Research literature on peer assessment provides a mixed 
view of its effectiveness (Wilson et al. 2015; McConlogue 2012; Kim 
2009). Students frequently complain about the quality of their peers’ 
marking (Wilson et al. 2015; Cartney 2014; McConlogue 2015). Other 
issues reported in the literature are friendship marking and collusion, 
poor quality feedback from lazy peers, incorrect feedback and unreliable 
marking with a wide variation in marks (Wilson et al. 2015; Cartney 
2014). 

Many of these problems occur because students are not in a 
position to make good quality judgements about their peers’ work, and 
lack competency to make reliable judgements. Preparation is essential 
in order to develop competency and integrity (Carless 2013) in peer 
assessors. This can be done through staged activities planned across a 
programme of study; students can begin to develop an understanding of 
academic standards on the programme and can then more reliably judge 
the quality of their own and others’ work (see Chapter 7 for details). 


Academic Literacies 


The writing in the disciplines (WID) movement in the US (Crossley et al. 
2017; Russell 2002) and academic literacies scholars in the UK (Lea and 
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Street 1998; Lillis et al. 2015; Lillis and Scott 2007; Hyland 2009; Ivanič 
2000) have studied the complexities of writing in higher education. When 
students begin their studies, they are entering a disciplinary community 
with its own, unique discourse. Disciplines have specific epistemo- 
logical understandings, sometimes these overlap with other disciplines 
and sometimes they are contradictory. Students need to become part 
of this disciplinary community through practising the specific writing 
genres used to communicate e.g. lab reports, briefing reports, case 
studies, essays, reflective journals, blogs. Each of these genres have 
particular rhetorical conventions; for example, in some disciplines 
the use of the first person T is avoided, and the passive voice is used 
extensively while in others the first person is used extensively, especially 
in reflective journals. These genres and their rhetorical conventions may 
not be familiar to students, especially students from non-traditional 
backgrounds, so it is important that programmes of study are planned to 
provide opportunities to practise and learn disciplinary discourse. 

Within academic disciplines there are established ways of 
thinking, investigating, creating and communicating knowledge 
(Entwistle 2005). As expert members of their discourse communities, 
academics are skilled in ways of thinking in their discipline and in 
communicating this thinking. Research on academic literacies has 
provided evidence of disciplinary differences in the way knowledge is 
constructed and communicated. The academic’s view of what counts 
as evidence, how arguments are constructed, how analysis is carried 
out and what counts as criticality, are all influenced by the underlying 
epistemological belief system of the discipline. An example of how 
this affects students comes from an empirical study in a widely cited 
paper by Lea and Street (1998). Lea and Street present a case study of 
a first-year student who received widely different grades for his essays; 
one in history and one in anthropology. For the history essay, he drew 
on his previous experiences of writing school essays and received a 
good grade and feedback. For the anthropology essay, a new discipline 
for him, he received a fail grade and comment that he needed help with 
writing and structuring an argument. The authors make the point that 
the different grades don’t reflect the student’s writing skills, but rather 
the different epistemological stances of the markers. Learners need 
guidance to enter particular discourse communities, understand ways 
of thinking within a discipline and how to communicate that thinking 
within disciplinary rhetorical conventions. 

Writing is a valuable tool for developing thinking. Through short 
writing tasks and writing-to-learn tasks, students can explore disciplinary 
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ideas and practice ways of expressing these ideas in writing. Short 
writing tasks can help develop student thinking and writing, without 
creating additional work for the teacher (QMUL 2019). The Thinking 
Writing website has examples of short, writing-to-learn tasks which can 
be used in lectures, seminars and online discussion fora. For example, in 


Teaching ideas — Writing-to-learn tasks 


The Thinking Writing website suggests these writing-to-learn tasks 
‘for introducing writing to lectures and seminars: 

Before the start of a lecture or seminar ask your students to write 
down the main points they learnt in the previous week and what they 
hope to learn today. 

Stop in the middle of your lecture and ask your students to 
formulate in a couple of sentences what they think is the main point 
so far, or a question/confusion they have in their minds. 

Vary the kind of prompts you give, so that students have to do 
different kinds of thinking. For example, ask students to write for 
a minute or two about their own experiences of some concept or 
phenomenon the lecture or seminar is covering. This may help make 
connections that help to build understanding. 

At the end of your lecture/seminar ask students to summarise 
what they feel they have learnt or pose a question. Collect a sample in 
or ask individuals to read theirs out — the students have had to think, 
and you've had valuable feedback which may alter what you do next 
time you meet. 

Writing can help students to explore concepts and identify 
what they know and what they need to learn more about. Getting 
students to write means they externalise what they know (or, as 
usefully, recognise what they don’t know) and how they are thinking, 
in writing. Once they have done this, they can do things with what 
they’ve written: look back at it, speak about it, compare it, shape and 
extend it. 

Ask your students to pick out a key phrase or word in what 
they’ve written, put it at the top of a new page, and write for a further 
two minutes on that phrase or word. This will extend their thinking 
and begin to give shape to their initial thoughts.’ 


(The Thinking Writing website, QMUL 2019) 


KEY PERSPECTIVES 


21 


a lecture you can stop at an appropriate point and ask students to spend 
a couple of minutes individually writing down the main points of the 
lecture. Often lectures introduce a lot of new content and processing this 
information can be demanding for students. A short pause during which 
they have time to reflect, remember and formulate understandings of the 
main points, helps to consolidate learning (see box on previous page). 

For students, moving from short writing tasks to longer examples 
of disciplinary writing, should involve the teacher designing and setting 
short writing assignments. For example, Troy et al. (2016) suggest tasks 
such as these in engineering as an alternative to longer lab and project 
reports. The different tasks help prepare students for the kind of thinking 
and writing they will need for longer, more complex assignments. The 
tasks are categorised as: Explain a Concept, Explain a Problem, How 
Stuff Works, Real World Example, and Pick a Side (all quoted from Troy 
et al. 2016). 


Short Writing Tasks in Engineering 


Developing an Assignment 
EXPLAIN A CONCEPT 


For Explain a Concept questions, instructors are asking students to 
explain different terminology, equations, and theory in their own 
words. These types of problems typically involve minimal deep 
thinking on the students’ parts, and answers are often found within 
supporting texts such as textbooks. Example: ‘In your own words, 
explain the meaning of Bernoulli’s equation.’ 


EXPLAIN A PROBLEM 


Instructors looking for students to understand the reason behind 
solutions to problems can opt for assigning Explain a Problem 
questions. Explain a Problem questions are potentially the easiest 
to assign because they can be created simply by placing the words 
‘Explain’ or ‘Why?’ at the end of any calculation-based problem. 
Example: ‘Calculate the maximum velocity achieved by wheel A and 
wheel B. Explain why they are different.’ The format of such explan- 
ations can be as detailed or simplistic as the instructor desires. 
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How STUFF WORKS 


Assigning problems that ask students to explain how different objects 
work helps students link classroom concepts to concrete objects, in 
turn reinforcing the real-world applications of their learning. How 
Stuff Works problems require students to understand concepts in more 
detail compared to previously discussed writing questions. Example: 
‘How does an air compressor work?’ The instructor can prompt 
students with concepts to include or leave the problem open-ended. 


REAL WORLD EXAMPLE 


Similar to How Stuff Works problems, a Real World Example question 
requires students to link theory and concepts they have learned in class 
to objects they interact with on a regular basis. The level of learning 
required for Real World Example problems is higher than Explain a 
Concept or Explain a Problem, and help students connect knowledge 
to practice. Example: ‘Explain how the water distribution system of 
a tall building works and use a specific example to help illustrate.’ 
These questions are effective at linking classroom material to actual 
examples, which is an important motivator for student learning. 


PICK A SIDE 


Pick a Side problems require students to link information learned in 
class to real world situations, but also requires them to take learning 
a step further and defend a particular side. Typically, these questions 
look at engineering problems from a broader perspective like the 
environmental, societal or political viewpoint, and as such are good 
questions for addressing learning outcomes beyond the technical 
concepts. Example: ‘Choose an environmental, economic, or societal 
standpoint, and argue for or against the construction of the Keystone 


XL Pipeline. (Troy et al. 2016) 


Research-Based Learning 
Research-based learning (RBL) covers a range of practices, namely, 


inquiry-based learning, object-based learning, case studies or scenario- 
based learning and problem-based learning. These activities can be 
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more or less structured depending on the programme and the expertise 
of the teachers and students involved. Through well-designed activities 
involving research, students learn about knowledge building and 
creation in their discipline, and how disciplinary knowledge affects their 
profession and their communities. 

There are a range of RBL practices; some focus on requiring 
students to learn research skills, while others are more ambitious and 
aim to initiate students into the disciplinary community as an equal 
knowledge builder (Brew 2006). Levy and Petrulis (2012) propose modes 
of RBL ranging from identifying to authoring. Identifying characterises 
work at the start of a programme where students will be carrying out 
research tasks designed by the teacher with the aim of exploring existing 
knowledge. Authoring characterises research tasks where students are 
involved in designing their own studies, critiquing existing knowledge 
and are engaged in knowledge building. Typically, authoring research 
would be carried out by final year students or postgraduate students, but 
this is not exclusively true. It can be difficult for students to make much 
progress in a research study in the space of a year; Chang overcame this 
limitation by developing an ‘inheritance mechanism’ (2005, 391); this 
entailed students documenting their research and findings and then 
handing these over to the next cohort. In this way, the next group makes 
more rapid progress, adds to the previous group’s work, and over a 
period of years there is accumulation of data and knowledge that leads 
to expertise in the field. Chapter 5 discusses RBL more fully, provides 
example tasks and a discussion of the advantages and challenges of 
designing RBL. 


Validity and Reliability 


A valid assessment measures what it claims to measure at the appropriate 
level. Validity is linked to the aim of the programme and the level of 
programme. If students have learned about X, and are then tested on 
Y, the test is not valid. Reliability means that assessors make similar 
judgements about the quality of students’ work; reliable assessors have 
a common understanding of the standards students are required to meet 
and how these standards are represented in the assignment. With this 
understanding they are able to make similar judgements about student 
performance. 

Validity and reliability are often at variance. Setting a task which 
requires students to be creative, critical thinkers, and then assessing it 
through multiple-choice questions which focus mainly on factual recall, 
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is an example of an assessment that is high in reliability but low in 
validity. The type of assessment task and complexity of learning affect 
validity and reliability. More complex tasks are often high in validity 
as they enable students to demonstrate higher order achievement, but 
assessing these tasks is more problematic. Judgements on the quality of 
students’ work are often subjective and less reliable. 

The assessment tasks in a module need to be designed so that 
they provide an opportunity for students to learn, show what they have 
learned and demonstrate that they have achieved competency in the 
knowledge, skills and becoming the module covers. If assessors believe 
the assessment tasks provide these opportunities, then the tasks are 
seen as having validity; similarly, students perceive tasks as valid if they 
provide them the opportunity to explore their understandings of module 
knowledge and skills. 

Designing valid tasks can be tricky and difficult to balance with 
the need for reliability in assessment judgement (see Chapter 5); in 
general, the greater the task validity, the less reliable the assessment 
judgements. The nature of learning in higher education typically requires 
assessment tasks to be open-ended, ill-structured tasks (Voss and Post 
1988), which provide opportunities for students to demonstrate higher 
order thinking, such as synthesis, analysis, argument and criticality. 
Ill-structured problems are ‘open’ tasks with no one correct answer. For 
example, a well-structured task such as converting 50 pounds to dollars 
at the current rate of exchange would have one ‘correct’ answer, so the 
assessors’ job is fairly straightforward. Assessing student responses 
to an ill-structured problem, such as ‘What was Einstein looking for 
when he discovered relativity?’ [Im indebted to Professor David 
Dunstan, Professor of Experimental Physics, Queen Mary University 
of London for this question] is less straightforward. Answering the 
question requires analysis, judgement and creation of an argument 
with supporting evidence. Assessment of ill-structured tasks is far less 
reliable than assessment of well-structured tasks (see Chapter 5 for a 
detailed discussion of marker reliability). Typical assessment tasks in 
higher education, such as essays, reports, oral presentations, objective 
structured clinical examination and project-based learning are all ill- 
structured tasks. 

Studies on marker reliability in higher education have found 
evidence of huge variations in assessor judgements (Williams and Kemp 
2019; Bloxham et al. 2015; Tisi et al. 2013). Williams and Kemp inves- 
tigated agreement between internal and external markers of master’s 
dissertations. Of the 324 theses grades investigated, only 9 per cent 
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required moderation. However, most theses were awarded B+ to A 
grades. The small range used makes agreement more likely. They argue 
that assessment criteria do not make marking more reliable and that 
‘there simply is no objective way to determine the academic value of a 
thesis’ (Williams and Kemp 2019, 769). 

One reason advanced for this variation is that assessment tasks 
are ill-structured, and a range of responses are possible, making reliable 
judgements more difficult. Ways of increasing reliability centre around 
developing documentation of standards through explicit assessment 
criteria and the use of rubrics in order to pin down what markers should 
be attending to when making a judgement. Documenting standards 
involves writing standards in explicit language. Rubrics contain both 
assessment criteria and a description of levels of attainment. Panadero 
and Jonsson (2013, 142-3) provide an example of a rubric for summary 
writing. For one criterion, ‘secondary ideas’, high-level attainment 
is described as expressing ‘important ideas ... with an economy of 
words’. In the next level down, there is an expectation that students will 
express ‘important ideas ... but with an excess of words’. Markers need 
to interpret what an ‘economy of words’ and an ‘excess of words’ looks 
like in students’ work. Similarly, for the ‘precision of terms’ criterion, 
high-level attainment is described as choosing words that are ‘efficient 
and concise’ while at the next level ‘words chosen are to some extent 
concise’. The rubric makes use of ‘hedge words’ (Sadler 2014, 279) in 
describing levels. For example, hedge words such as ‘to some extent’ and 
‘more or less’ are used. The marker needs to make a judgement as to what 
‘to some extent’ would look like in students’ work. The aim of the rubric 
is to make marking more reliable and standards more explicit, but the 
fuzziness of the hedge words thwarts this aim. 

Research shows that markers interpret rubrics and assessment 
criteria differently. For example, critical thinking features strongly in 
assessment in higher education, but interpretations of what is good or 
poor quality critical thinking differ. Studies show that markers interpret 
criteria differently, ignore criteria and add their own personal criteria 
(Bloxham et al. 2011). There is also evidence that markers make holistic 
judgements about the quality of student work and then use criteria 
to retrospectively justify their judgements (Grainger et al. 2008). 
Moreover, Sadler argues that the nature of learning in higher education, 
and the complex tasks that students are involved in, makes it impossible 
to tie down standards into a small set of explicit assessment criteria 
and level descriptors (Sadler 2014). Judgement draws on a vast range 
of tacit knowledge (Polanyi 2009) drawn from previous learning and 
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previous experience of marking. This rich bundle of tacit knowledge is 
particular to each marker, so it is important that markers have oppor- 
tunities to explore and compare their judgements. Standards are held 
within disciplinary communities and it is only through discussion in 
these communities that markers can reach agreement on standards. 
Watty et al. (2014) describe a calibration activity where a disciplin- 
ary community came together to discuss exemplars of students’ work, 
compared their judgements and tried to reach consensus. There is 
increasing research on calibration as a way of establishing academic 
standards within a discipline (Watty et al. 2014; O’Connell et al. 2016). 
Chapter 6 discusses ways of bolstering marker reliability. 


Follow-Up 


This chapter provides an overview of key ideas in higher education; 
there are many ideas to follow-up. Choose an idea you are interested in 
and read the relevant chapter, which has further ideas for reading and 
follow-up activities. 


Reading 


Explore Wenger’s writing on communities of practice. Start with Wenger, 
Etienne. Communities of Practice: Learning, Meaning, and Identity. 
Cambridge: Cambridge University Press, 1999. 


Thinking 


Think about academic standards on your programme. How are standards 
shared within the programme team and with students? 


Investigating Your Practice 


Investigate writing on your programme. How many different genres are 
students required to write? What preparation do students receive? Are 
there any writing-to-learn tasks or short writing assignments? Are there 
opportunities to submit and receive feedback on writing drafts? 

Investigate which assignments students find most difficult and why. 
Ask final year students to tell you which assignments they struggled with 
and what caused them difficulties. 
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Part 1 
DESIGNING ASSESSMENT 


Chapter 3 


Designing Assessment for a 
Course Unit/Module 


This chapter will help you think through the process of assessment 
design for your course unit or module. A course unit or module is a 
discrete unit of study, involving teaching time (either face-to-face or 
online), individual study, possibly group work and typically at least one 
summative assessment. In the UK the size of modules vary, some may 
run throughout the academic year while others may be short, one-week 
intensive courses. Regulations stipulate how many hours of learning 
(and thus how many modules) need to be completed for a degree 
programme. Learning hours are measured in credits; in the UK, fifteen 
credits are equivalent to 150 notional hours of learning (this includes 
taught sessions, self-study time and time spent on assessments). A 
bachelor’s degree with honours is typically equivalent to 360 credits and 
a master’s degree, 180 credits in England, Wales and Northern Ireland 
(see QAA 2014). Modules are combined to make a degree programme. 
There can be considerable flexibility in how modules are combined 
and a wide range of choice for students in some programmes. In other 
programmes of study, typically professional programmes, there may be 
many compulsory core modules with very little choice. Whether you are 
working in a modular system or not, you will be involved in designing 
course units and appropriate assessments. Early career teachers are 
often asked to design new course units/modules, and once they gain 
more experience, may be involved in programme design. For this reason, 
the chapter on course unit/module design comes before the chapter on 
programme design (Chapter 4) though you are encouraged to read both 
chapters. This chapter covers the following. 


31 


32 


e Backward design to help you think about what you want 
students to learn from the course unit/module and how the 
course unit/module aligns with others in the programme 
(see Chapter 4 for programme-level assessment design). This 
chapter proposes that you begin by thinking about what you 
want students to know, do and become as a result of the course 
unit/module and to consider how the course unit/module fits 
in with and supports student learning across the programme. 

e The concept of validity, ensuring that the assessment tasks test 
the aims of the course unit/module, is discussed and strengths 
and weaknesses of different task types are compared. 

e Ways of developing clear assessment guidance and communi- 
cating these to students are discussed. 

e An introduction to designing authentic assessment tasks and 
tasks that involve external participants (more detail can be 
found in Chapter 5). 

e Assessment design should include setting formative tasks 
which help students develop the knowledge and skills 
they need to tackle summative assessments. This involves 
developing writing and speaking and designing opportun- 
ities, which give students practice in the assessment genre and 
appropriate rhetorical conventions (see Chapter 2 for more on 
academic literacies). 


What Should you Assess and How Should you Assess It? 


Start with a clear idea of what you want students to learn in your 
course unit/module and how their learning links with learning in the 
programme. In many fields, content dominates, and syllabi are often 
packed full of content that students are expected to learn and memorise. 
Assessment of this learning then becomes a matter of checking that 
students have a good knowledge of all this content; typically, in content 
heavy programmes this is done through multiple-choice questions or 
short answer questions, especially in science, technology, engineering and 
mathematics (STEM) subjects. However, these methods are limited when 
it comes to assessing higher order thinking and complex learning, and the 
construction of valid multiple-choice questions requires expertise. 

In Chapter 2, complex learning was discussed. In higher education, 
learners are concerned with critically evaluating theoretical papers 
and research findings, with constructing coherent arguments and 
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understanding professional practice protocols. Crucially, learners are 
expected to become part of a disciplinary and professional community 
(Sambell 2013; Barnett 2009), understand ways of thinking and 
practising (WTP) within a discipline (Entwistle 2005) and understand 
the rhetorical conventions used to express that thinking in spoken and 
written language (Bean 2011). Barnett expresses the belief that higher 
education can be transformational; learning in higher education is not a 
matter of memorising content, acquiring knowledge and skills; it is a‘... 
vehicle for effecting changes in human beings ...’ (2009, 432). He views 
the curriculum, not as a series of content to be learned, but as transform- 
ational. Knowledge in the modern world is not static; knowledge is 
contested and open to change. Barnett describes the world as ‘super- 
complex’, open to manifold interpretations which create uncertainty 
and insecurity (2009, 439). Having knowledge and skills does not help 
an individual cope with this supercomplexity, but, through personal 
development and reflection, students’ world view can be transformed. 
As they become members of a professional or knowledge community 
they develop their sense of being in the world and their understanding of 
their profession becomes based on ‘well-founded claims’ (Barnett 2009, 
432) arrived at through a critical stance. 

Entwistle comments on the importance of recognising the 
limitations of learning outcomes and designing open-ended assessments 
(or ill-structured tasks, Voss and Post 1988), to capture complex learning. 


While WTPs, by their very nature, are more difficult to assess, 
limiting the assessed outcomes to more precisely defined outcomes 
is potentially damaging to students’ understanding of the subject 
itself. The broader understanding of the subject can be encouraged 
through more open-ended forms of assessment and through 
ensuring that students cover general or problem-based questions 
or assignments. (Entwistle 2005, 79) 


A syllabus cannot be a description of course unit/module content. In 
some fields, such as medicine, the content to be covered has increased 
exponentially; adding more content to a saturated syllabus is not an 
option. Focusing on developing the learners’ capacity to understand 
how members of the academic and/or professional community think 
and how knowledge is created within that discipline, enables students 
to critique and deconstruct knowledge. Learners are enabled to indepen- 
dently explore areas of interest in the discipline, equipped with the tools 
needed to understand, question and construct new knowledge. They 
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become a part of the disciplinary community, contributing to knowledge 
building in the discipline and/or in professional practice. 


Backward Design 


Assessment is a fundamental part of learning and has more impact on 
student learning than teaching. Assessment ‘... directs attention to what 
is important. It acts as an incentive for study. And it has a powerful effect 
on what students do and how they do it’ (Boud and Falchikov 2007, 3). 
But often assessment is the last item to be considered in course unit/ 
module design. Backward design ensures better connections between 
assessments by starting with the last assessment task and planning 
teaching activities around assessments. The earlier assessments are 
‘scaffolding assignments’ (Bean 2011, 96) providing students opportun- 
ities to develop the knowledge and skills needed for the final assessment. 
Assessment tasks need to be motivating, relevant and create opportun- 
ities for learning. As Graham explains: 


With this approach to planning, teachers begin with a clear 
statement of goals and design assessments to determine student 
progress toward those goals. The selection and organization of 
activities, texts, and materials into daily plans come last in the 
teacher’s thinking process. Through this lens, assessment provides 
opportunities for students to demonstrate their learning in 
productive ways, including, but not exclusive to, final performances 
based on complex learning goals. (2005, 609) 


The planning process starts with an ‘overarching question’, and 
assessment is designed to explore this question. The overarching 
question and learning activities need to be appropriate for the level of 
study. Teaching and learning activities support learners to complete the 
assessments. Backward design aims to ensure that assessment is fully 
integrated into the course unit/module and programme. 

Having worked out what you want students to learn, the next step 
is to design a syllabus to support that learning, thinking and communi- 
cating, with valid assessment tasks that enable students to further their 
learning and share what they have learned. For example, in a first year 
undergraduate political analysis course unit/module, the goal was to 
help students understand the fundamentals of political analysis and 
to prepare them to write a fine-grained analysis of political texts. The 
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overarching question was ‘What is Politics?’. Assessment was through 
an essay. Learning activities involved analysing papers presenting 
contrasting views of politics, discussing these views in seminars, writing 
summaries of key papers and writing-to-learn activities (Chapter 2). To 
gain an understanding of standards on the course unit/module, students 
marked three essays from a previous year and discussed grades and 
comments. Writing activities, including synthesising ideas from a range 
of sources using a reading grid, prepared students for expressing their 
ideas in appropriate disciplinary discourse. A draft essay, drawing on the 
texts studied in seminars, was submitted and feedback provided by the 
seminar leader. Students had the opportunity to ask questions about 
the feedback and areas for development before submitting their essay for 
summative assessment at the end of the first term. (This video shows the 
course unit/module leader talking about fine-grained analysis and using 
an exemplar student text to illustrate his talk: http://www.thinkingwrit- 
ing.qmul.ac.uk/node/196.) 

Scaffolding tasks guide students’ thinking and help initiate students 
into ways of thinking in the discipline. Thinking Writing provide a series 
of worksheets guiding students’ reading of literature in geography (see 
below). The first worksheet, reproduced in the following box (Thinking 
Writing 2019), requires students to identify key sentences in an article 
and also sentences they cannot understand. The worksheet gives practice 
in paraphrasing and citing references; students practise expressing their 
opinion of the article in disciplinary discourse. 


Geographical Ideas in Practice 
Worksheet 1: Reading and discussion 


InTeL EPEE EEA AA AE EA AEE EAA NAE 
MUS ACE AA T ENAA E NETET EE IEE E A 
Reading (s): odenis a E EE E E 


Read the article suggested by your tutor and complete the following 
activities. 


1. Make a list of the key terms in the field; make sure you know what 
they mean as you will be discussing them in the next tutorial. 


2. After/while reading the article, write down the following (your 
tutor will give you one or two of these): 
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e the sentence/paragraph that you feel is most important in 
the article 

e the sentence/paragraph that you disagree with most, or feel 
has the most problems 

e the sentence/paragraph that you just cannot understand or 
see the significance of 

e the sentence/paragraph that you like the most 

e the sentence/paragraph that you feel has the most striking 
piece of data/information in it. 


In addition to this, include the citation and the full reference for the 
article in the correct form. 


3. Rewrite the sentence/paragraph in your own words, again using 
a citation in the text. You do not have to repeat everything from the 
original, but you do have to cover the main points. 


For example: 


Watson (1980, 357) believes that the majority of people find 
writing hard. 

or 

The majority of people appear to find writing hard (Watson 
1980, 357). 


4. Write a paragraph of five to six sentences that explain why you chose 
this section, and why you feel it is most important/has problems/is 
most striking/etc. If you can, try not to use the words ‘I think ...’. 


‘Writing — in the sense of producing texts composed of written words — 
seems to be difficult for most of us.’ (Watson 1980, 357) 


References 


Watson, P. ‘Conformity and commitment in writing’, Visible Language, 

14 (1980): 351-63. 
Source: Worksheet from Thinking Writing. 
www.thinkingwriting.qmul.ac.uk 
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The Trouble with Intended Learning Outcomes 


Intended learning outcomes (ILOs) describe the knowledge, skills and 
competencies which are to be learnt in the course unit/module. They 
attempt to describe what the student will be able to know and do, and 
to what level by the end of the course unit/module. The allure of ILOs is 
the assumption that learning can be precisely described, and assessment 
tasks designed to measure to what extent students have achieved the 
outcomes. But, from what we have discussed about the nature of learning 
in higher education (Chapter 2) and the nature of students’ performance 
and judgements about the quality of that performance (see Chapters 2 
and 5), such precise ‘codifications’ (Sadler 2014) and specification of 
learning and student achievement appears to be limiting and impossible. 
Hussey and Smith (2008) point out that ILOs have become part of 
a process of monitoring and evaluating higher education. However, 
the nature of learning in higher education, the complex learning that 
students are involved in and the ill-structured tasks (Voss and Post 
1988) that are necessarily set, make precise specifications of learning 
outcomes difficult. Hussey and Smith (2008) argue that ILOs are part of 
a process that tends towards the commodification of knowledge (2008, 
221), but contend that learning in higher education cannot be neatly 
commodified and specified. Similarly, Rotthoff argues that in medical 
education, competency-based education which depends on ILOs is inap- 
propriate because it is, ‘... poorly suited to highly skilled professions 
[...] such as doctors because they require highly complex skills such as 
analysis, judgment and reflection, professionalism and empathy. These 
[...] cannot be adequately achieved with the predominant methods of 
didactic learning objectives’ (2018, 1). A narrow focus on competencies 
is detrimental as medical students need ill-structured assignments in 
order to develop these complex skills. 


Motivating Learners Through Authentic Assessment 


‘Authentic assessment’ has been coined to describe assessment tasks 
which link knowledge with the kind of problems we face in society 
(Villarroel et al. 2018). Authentic assessment aims to help students 
develop strategies for applying knowledge and solving problems. 
Authentic tasks may take the form of scenario-based learning, case 
studies, research-based learning, object-based learning and problem- 
based learning (see Chapter 5). Authentic tasks may be carried out in 
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groups to replicate teamwork in professional and community contexts. 
Authentic assessment tasks, when well-designed, are ‘inherently 
meaningful and relevant’ (Sambell et al. 2013, 12) and have long-term 
value fitting Boud and Falchikov’s (2007) description of sustainable 
assessment. Authentic tasks may involve students producing work which 
is not viewed as academic but has a clear link with professional practice, 
such as information leaflets for patients on health care issues, videos of 
case studies for psychiatric practice, briefing reports for charities and 
non-governmental organisations and mathematical teaching resources 
for teaching maths in schools. Other tasks may involve students in more 
typical academic work, such as creating annotated bibliographies, collab- 
orating in writing articles for journal publication, or a book chapter or 
edited book. Chapter 5 has a full discussion of ways of involving external 
bodies in assessment and further examples of authentic assessment. 

These innovative assessment tasks can be hugely motivating 
for students but can also cause stress. New forms of assessment (new 
genres) which are unfamiliar to students can cause anxiety. Students 
prefer familiar assessments; Iannone and Simpson (2017) discuss the 
influence of discipline and institutional contexts on students’ perceptions 
of summative assessment. In a mixed-methods study of education 
and mathematics students in the UK they found that mathematics 
students preferred closed-book exams over projects, while education 
students preferred projects and dissertations, which they considered to 
be better at distinguishing ‘those who are good at academic studies in 
education from those who are poor at academic studies in education’ 
(Iannone and Simpson 2017, 789). Both mathematics and education 
students thought multiple-choice exams to be a poor discriminator of 
ability. They concluded, ‘In short, students in our studies have distinct 
epistemic beliefs and these appear to be one of the factors influencing 
their perceptions of assessment, and these are grounded in disciplinary 
differences’ (Iannone and Simpson 2017, 798). 

Table 3.1 sets out some common assessment methods and their 
advantages, disadvantages and disciplinary match. In choosing an 
appropriate assessment, you need to think about: 


- the time taken to plan and design the assessment 

- the knowledge, skills and competencies you want learners to 
develop 

- the disciplinary epistemic beliefs which influence assessment 

- the familiarity of the assessment to students 
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- the ease of administration 
- the workload for both students and teachers 
- the amount of preparation and practice students need (this 

increases with lack of familiarity) 
- the validity of the tasks 
- marking reliability (see Chapter 6 for a discussion of academic 

standards and marker reliability). 


Table 3.1: Advantages and disadvantages of types of assessment 
(adapted from Epstein 2007) 


Assessment | Marker Advantages Disadvantages Typically 
type reliability used in 
Multiple- High Has potential to Difficult to write Medicine, 
choice assess analysis good quality stems | health, law 
questions/ and problem- and distractors. and the 
Single best solving but often | Encourages sciences 
answer used to assess guessing. 
questions memorisation of | Difficult to assess 
‘facts’. complex learning. 
Quick to mark. 
Can use software 
to mark and set 
question. Can 
build up a question 
bank. 
Provides fast 
feedback and 
can be done 
formatively online 
with feedback. 
Oralexam [Low unless | Good for checking | Can be time All disciplines 
(viva) markers knowledge and consuming 
engagein | clinical reasoning. | and difficult 
calibration | Often used to deter | to organize, 
activities | plagiarism. especially with 
(see Can be good large classes. 
Chapter 5) | for synoptic Risk of hidden bias 
assessment, (gender, race etc.). 
connecting several | Examiners need 
modules. professional 
Can record development; 
and review often two 
to moderate examiners 
and check on required. 
standards. Can be extremely 
stressful for 
learners. 
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Assessment | Marker Advantages Disadvantages Typically 
type reliability used in 
Essay Low, unless | Depending on ‘Essayist literacy’ | Arts, 
markers type — explanation, | (Lillis 2001) may | humanities, 
engagein | argument, disadvantage social 
calibration | expository. students from sciences, 
activities | Good for testing marginalised some sciences 
(see analysis, synthesis, | groups. 
Chapter 5) | argument, An essay is far 
criticality. removed from 
Essay questions professional 
can be quick to set. | practices — reports 
Some students may have more 
may be familiar validity. 
with the genre. 

Lab report | Low, unless | Can provide Can be difficult to | Engineering, 
markers practice in set up with large sciences 
engage in | devising/ groups. 
calibration | understanding a Time in laboratory 
activities research question, | is expensive. 

(see collecting 

Chapter 5) | and analysing 
data, writing a 
discussion section. 

Observed Low unless | Examining Can be time Professional 

Structured | markers how learners consuming and fields — 

Clinical engage in | apply skills in expensive to set medicine, 

Exam calibration | professional/ up. legal practice, 

(OSCE) activities clinical settings. Requires several social work, 
(see Good for observing | examiners. psychology 
Chapter 5) | communication Can be stressful for | and 

skills. some learners. management 
Can be quick to 
mark. 
Scenarios, | Low unless | Allows learners to | Can be difficult Engineering, 
problem- markers show application | to design medicine, 
based engagein | of knowledge, authentically. education, 
learning and | calibration | teamworking and _ | Learners need social work 
case studies | activities | problem-solving support and —most 
(see skills. practice if the professional 
Chapter 5) | Can be good genre is unfamiliar. | fields 
for synoptic Can be complex 
assessment, and time 
connecting several | consuming to set 
modules. up and run. Can 
involve multiple 
assessors so high 
workload for staff 
and potentially low 
marking reliability. 
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Assessment | Marker Advantages Disadvantages Typically 
type reliability used in 
Portfolios Low unless | Allows students Can be very time- | Used in most 
markers to collect and consuming for professional 
engage in | curate learning both learner and | fields 
calibration | experiences. marker. 
activities | Good for showing | Can involve 
(see attainment of learners in 
Chapter 5) | professional a variety of 
and clinical assessment tasks 
competencies. —this may be 


Allows assessment 
choice so good for 


bewildering for 
some learners who 


inclusivity. will need support 
Can be good with new genres. 
for synoptic Potential for 
assessment, plagiarism but 


connecting several | can be combined 
modules. with an oral 
examination. 


Preparing Students for Assessment 


When introducing new varieties of assessment, it is important to prepare 
students and provide them with practice in any new variety so that 
they have, as Gibbs and Simpson (2005) state, a chance to get good 
at a particular variety before they are summatively assessed. Think 
about where, in your programme, students will have opportunities to 
practise assessment tasks and written genres as an integral part of their 
learning. With the massification of higher education in many countries 
worldwide, and the increasing mobility of students, the higher education 
teacher cannot assume that their diverse student cohort has previous 
knowledge of any form of assessment, or any clear understanding of 
the academic standards on a course unit/module. Even if students have 
had experience of oral presentations or writing lab reports and essays 
at school, the format and expectations at higher education institutions 
may be completely different. So, the first stage is to prepare students, 
especially first year students, for assessment. For an example, see the 
description of the political analysis first year course unit/module earlier 
in this chapter. 

Students need a good understanding of assessment processes, and 
especially of academic standards on the course, in order to evaluate 
and direct their learning. In Chapter 2, I discussed assessment literacy 
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(and see also Chapter 7), in this section I want to focus on one aspect 
of assessment literacy: understanding academic standards. An effective 
way of helping students understand the standard of work they need to 
produce is through the analysis of exemplars (Hendry and Tomitsch 
2014; Carter et al. 2018; Boud et al. 2018). 

Teachers develop an understanding of academic standards in their 
discipline through seeing the range of work that students produce; 
Sadler (1989, 2009, 2010) proposes that students need to be given 
similar experiences to develop their understanding of standards. When 
teachers see a range of work, they begin to develop an understand- 
ing of what students can produce and what good quality work looks 
like. Discussing student work with colleagues can help teachers to 
develop consensual understanding of what good quality work looks like 
(Sadler, 2010). Academic standards are held to be socially constructed, 
constructed and held by the disciplinary community (Sadler 2014). 
As discussion of a range of exemplars helps teachers understand the 
academic standard for their programme, it is proposed that exposing 
students to a similar process will help them develop an understanding 
of academic standards (Sadler 2010). But the use of exemplars is not 
standard practice on academic programmes. Model answers may be 
given to students, but exemplars of similar assignments from a previous 
year, showing a range of students’ work (and the range is important) is 
less common. Some students report not seeing examples of their peers’ 
work during their studies. In these cases, the student is in the dark about 
what good quality work looks like; explicit ways of conveying standards, 
such as explicit assessment criteria have been shown to be ineffective 
(see Chapter 5). Students need to see examples of a range of their 
peers’ work, in order to gain an understanding of standards on their 
programme. In large, diverse classes, where students come from a range 
of backgrounds, some non-traditional, exemplars are especially useful 
in helping students understand academic standards. For example, 
health care professionals who return to education after a break of many 
years, may find academic tasks, such as short answer questions, report 
writing, oral presentations and literature reviews, unfamiliar and may 
be uncertain about the quality of work they need to produce. Similarly, 
students from non-traditional backgrounds, and mature students, may 
have experienced a limited range of assessment types — for example, 
multiple-choice questions — and may struggle with other forms of 
assessment, especially the essay (Lillis 2001). 
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The essay has been extensively used in the arts, humanities and 
social sciences. Academic literacies researchers have criticised the prolific 
use of the essay in assessment in higher education as it disadvantages 
students entering universities from non-traditional backgrounds. Lillis 
and Tuck explain that ‘academic literacies researchers have argued that 
the entrenched privileging of essayist literacy perpetuates inequalities 
in the academy, closing down diversity in knowledge-making, working 
against policy goals of widening access’ (2016, 6). 


Diversity and Inclusion 


Widening participation has created greater diversity in student cohorts; 
many more students are the first in their families to participate in higher 
education. Dymond-Green (2018) reports on Henderson and Shure’s 
tracking of entrants to higher education; their data suggests: 


[...] that nearly 15 per cent of the English population born in 1990 
are first in the family university graduates. This means that they 
comprise more than half of university graduates for this cohort 
(graduates who match their parents with a degree comprise nearly 
12 per cent of the population). There is some suggestion from 
our early analyses that this is driven by increased participation by 
women and black and minority ethnic groups. 


A greater variety of assessment types that link more with professional 
practice and better preparation of students for assessment is needed, to 
create more of a level playing field and allow students to demonstrate 
what they can do. However, there needs to be a balance as too much 
variety in assessment can bewilder students; they need practice in the 
selected range of assessments used on the programme. 


Reflective Writing (and Oral Reflection) 


In some disciplines, especially professional fields such as social and 
health care, the use of reflective statements has become widespread. 
Linked with the idea of reflective practice (Schön 1983; Moon 2004; 
Brookfield 2017), the reflective statement provides an opportunity for 
students to review their practice, often focusing on an incident and 
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analysing what happened, how they responded to the incident, how 
they felt about their response and what they learnt. The analysis acts 
as a springboard to self-assessing development needs and planning 
future learning to meet those needs. Reflection and reflective writing 
are widely advocated in professional development courses. However, 
assessment of reflection is more contentious (Bolton, 2010; Boud, 
1999; Rai, 2006; Ross, 2014); reflective writing is seen as performa- 
tive, lacking in authenticity, confessional and emotionally problematic, 
as students write about themselves and their emotions. Assessing this 
writing as unsatisfactory can seem to imply criticism of the person. 
When reflective writing is compulsory and summatively assessed, 
students tend to be strategic in producing what they think the assessor 
wants, rather than authentic reflection. Reflective writing assignments 
can ‘normalise surveillance of students’ emotional and developmental 
expression and produce rituals of confession and compliance’ (Ross 
2011, 113). 


Validity 


In designing assessment, the teacher needs to have a clear idea of what 
they expect students to learn and what level of knowledge and skills 
students are expected to achieve. This needs to be matched with an 
appropriate assessment task that will enable students to demonstrate 
their learning. The task needs to have validity, connect with the aims of 
the course unit/module and be feasible within the time restraints and 
practicalities of the course (see Chapter 2 for a full discussion). 


Writing Clear Assessment Briefs 


To learn effectively from assessment, students need a thorough under- 
standing of the assessment method(s). Developing a clear assessment 
brief, preferably with students, helps them to gain this understanding. 
An assessment brief details the task and gives students helpful guidance. 
It is important that the assessment method does not assume knowledge, 
skills and competencies that have not been covered in the syllabus; 
this would disadvantage some groups of students and give others an 
advantage. For example, do not assume that students have expertise in 
video production, oral presentations or writing genres such as reports or 
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essays. If this expertise is important, then learning how to make a video 
or write an essay needs to be part of the course unit/module and students 
need practice to acquire these skills. Co-constructing assessment tasks 
with students is an excellent way to ensure they have a thorough under- 
standing of the task (see Chapter 7 for more on co-construction of tasks 
and assessment criteria). 

Setting the same task each year invites plagiarism (Carroll 
2002). To design out plagiarism, teachers should devise authentic 
tasks which enable students to carry out similar tasks with different 
topics or give students, or groups of students, different scenarios or 
data sets to work with. For example, in the first year of a three-year 
undergraduate programme in biology, students worked in groups to 
discuss a key concept, namely, structure and function in biology. They 
began discussing the concept online, then in groups prepared a short 
presentation on structure and function of a group of organisms or single 
organelle (each group researched a different organism/organelle). 
This task involved literature searching and inclusion of a small number 
of references in the presentation. The group presentation was given in 
a seminar to the teacher and other group members. The presentation 
was not summatively assessed; the group received formative feedback 
from peers and the teacher. Following this, students were tasked with 
preparing a poster on structure and function in different organisms or 
an organelle. Posters were displayed and uploaded online; students 
peer reviewed and voted on the best poster. The next assessment task 
was to individually write a short essay on structure and function, 
choosing again a different group of organisms or organelle and 
including a short literature review. The draft essay was submitted for 
teacher feedback which was used to improve the final essay on structure 
and function. The final essay was assessed by the teacher as part of the 
summative assessment for the course unit/module (see STEM Wishees 
2019 http://www.thinkingwriting.qmul.ac.uk/wishees/collections/ 
queenmary/biologyundergraduateposter/55580.html for sample 
posters and teacher commentary). 

The group presentations, literature review, posters and draft essay 
prepared students for writing the final, summatively-assessed essay. 
They also provided practice in developing research skills, team working 
and exploring existing knowledge; students could draw on this learning 
to tackle more extensive research projects in subsequent course unit/ 
modules. Each group chose different groups of organisms or single 
organelle for each task. This demonstrates how different topics for each 
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task and each group, as well as thorough preparation, can help reduce 
plagiarism (Carroll 2002). 


Assessment Briefs 


Clear instructions for assessments are provided in an assessment brief. 
An assessment brief needs to provide all the information that students 
require in order to successfully tackle the assignment. Typically, 
assessment briefs will provide information on (adapted from Bean 2011; 
University of Suffolk 2019; University of Plymouth 2019): 


- workload and time allowance 

- weighting of the assessment, i.e. percentage of course unit/ 
module assessment 

- word count 

- accessibility and inclusivity (access to resources; field trips etc.; 
adaptions for statements of reasonable adjustments (SoRas), 
e.g. if the student cannot do fieldwork what alternative method 
will be provided?) 

- choice in assessment (can students choose to present their 
work in a presentation, video or essay? See Chapter 9) 

- clear description of the tasks and modes of working (e.g. in 
teams), if applicable 

- an explanation of how the task fits with the learning aim of the 
course unit/module, to establish validity 

- an explanation of how the task will be assessed 

- a link to work on exemplars to develop student understand- 
ing of assessment standards (see the description of guided 
marking in Chapter 7) 

- assessment criteria (but see the discussion on the limitations of 
assessment criteria in Chapter 5) 

- links between this assessment and others in this course unit/ 
module and subsequent course unit/modules across the 
programme of study 

- information on resources — access to equipment (laboratories, 
video equipment etc.) 

- bibliographies and access to set texts 

- contact details of external facilitators (industry partners, non- 
governmental organisation partners) 
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- contact details for help and guidance - drop in help, online 
FAQs, technical support (e.g. video technicians or web design 
support) 

- links to an online discussion forum for questions to peers and 
the teacher 

- submission dates and arrangements for drafts and feedback 

- submission date(s) when marks and feedback will be available 

- date and location of follow-up feedback workshops. 


Reviewing an Assessment Brief 


The questionnaire in Table 3.2 highlights the areas you need to think 
about when designing assessments and writing an assessment brief for 
students. Use this questionnaire to design a new assessment or to review 
the existing assessment brief for your course unit/module. 


Table 3.2: Assessment brief review questionnaire (sources: adapted from 
Bean 2011; University of Suffolk 2019; University of Plymouth 2019) 


Task design: validity 


alk Is the assessment valid? Does it link to the aims of the module? 


Comment: 


Action required: 


2. Does the assessment task(s) enable students to demonstrate that they 
have met these aims? 


Comment: 


Action required: 


3. Is the assessment task clearly explained? What opportunity is there for 
students to check their understandings of the task (e.g. virtual learning 
environment forum discussion, FAQs, exemplars of student work for 
similar tasks)? 


Comment: 


Action required: 


Academic standards 


4. Do students understand assessment standards on the module? How do 
you know? Have you discussed exemplars of previous students’ work or 


organised guided marking (see Chapter 7)? 
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Comment: 


Action required: 


5. Are academic standards on the module in line with ‘sector recognised 
standards’ (Quality Assurance Agency 2019)? Have you checked 
appropriate national documentation (e.g. the Framework for Higher 
Education Qualifications, QAA 2014, and Subject Benchmark Statements, 
see QAA 2019a)? 


Comment: 


Action required: 


6. Do students understand the assessment criteria? Is their understanding 
the same as yours? How do you know? (See Chapter 6.) 


Comment: 


Action required: 


7. Are students involved in designing the assessment or grading and giving 
feedback? Are they competent to take on these tasks? How do you know? 
(See Chapter 7.) 

Comment: 


Action required: 


Preparation 


8. Have the knowledge and skills required for this assessment been practised 
in formative tasks? 


Comment: 


Action required: 


9, Is there any aspect of the assessment that requires knowledge and skills 
that haven’t been covered in the curriculum (either in this module or a 
previous one)? If yes, what are they, e.g. results need to be presented 
in Excel®? Can this aspect be removed from the task or can students be 
given an alternative way of presenting data or can additional classes (e.g. 


Excel® classes) be included in preparation activities? 
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Comment: 


Action required: 


10. How does this assessment link to assessments in previous and future 
modules? What skills and knowledge from previous modules can students 


use to tackle this assignment? 


Comment: 


Action required: 


Inclusivity 


11. Does the assessment brief explain what support is available to students 
(e.g. office hours, virtual learning environment forum, student mentors, 
writing support etc.)? 


Comment: 


Action required: 


12. Does any element of the assessment exclude groups of learners, e.g. 
are field trips accessible? Do scheduling arrangements respect religious 
observances? 


Comment: 


Action required: 


13. Does the assessment brief explain what choice is available (e.g. choice of 
assessment tasks such as an essay or presentation, see Chapter 9)? 


Comment: 


Action required: 


14. Does the assessment brief explain any necessary alternative arrangements 
for students with SoRAs? [N.B. it is better to design assessment in an 
inclusive way so that no alternative arrangements are necessary. See 
Chapter 9 for a full discussion. ] 


Comment: 


Action required: 
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Assessment genre 


15. Does the assessment brief explain the audience for the task, e.g. public 
information leaflet, briefing report for local charity managers? Have 
students had practice in communicating with this audience? (See 
Chapter 5.) 


Comment: 


Action required: 


16. Have students had experience in your module or a previous module of 
the assessment genre, e.g. writing an essay, giving an oral presentation, 
presenting a case or mooting? If not, how can they get this experience? 
How can they get good at communicating in this genre before they are 
assessed (e.g. writing-to-learn tasks, see Chapter 2)? 


Comment: 


Action required: 


Mechanics 


17. Are the mechanics of the assessment clearly stated, e.g. submission date, 
word/page count (if appropriate), font size, presentation guidelines (if 
appropriate)? 


Comment: 


Action required: 


18. Do students have time to do the assignment? Are assessments ‘bunched’ 
or evenly distributed across the module and programme? 


Comment: 


Action required: 


19. Is plagiarism designed out of the assessment? Are tasks changed 
regularly? Are students prepared for tasks? Are they over-assessed, 
leading to stress and the impulse to plagiarise? 


Comment: 


Action required: 


20. When will students receive grades and feedback? Will they have time to 


act on this feedback to improve their next assignment? 
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Comment: 


Action required: 


21. What kind of feedback will students receive (see Chapter 8)? 


Comment: 


Action required: 


For teachers: reliability 


How will you ensure that feedback judgements (grades and feedback 
comments) are fair and consistent with the academic standards on the module 
(see Chapter 6)? 


Comment: 


Action required: 


Follow-Up 


Reading 


The University of Suffolk have produced useful guidance on assessment 
briefs and moderation of marking: 


https://www.uos.ac.uk/sites/default/files/assessment-overview. 
pdf 


The University of Plymouth has excellent resources on inclusivity, 
including guides and videos on inclusive assessment (and see Chapter 9 
in this guide for a detailed consideration of inclusive assessment): 


https://www.plymouth.ac.uk/about-us/teaching-and-learning/ 
inclusivity/inclusive-assessment 


Investigating Your Practice 


Work with a colleague(s) to review your course unit/module assessment 
guidance. Use the assessment review questionnaire to review the 
assessment brief and identify changes you would like to make. 
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Alternatively, work with some students to review the course unit/ 
module assessment guidance. These could be final year students who 
have an understanding of the structure of the course unit/module and/ 
or first year students (entry year students) who are new to the course 
unit/module and can give feedback on what they need clarified. Check 
their understanding is the same as yours by asking them to paraphrase 
key instructions in the brief. 
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Chapter 4 


Designing Assessment Across a 
Programme 


A Programme-Wide Perspective 


This chapter covers the challenge of designing assessment across 
a programme. It focuses, to a large extent, on modularised degree 
programmes. Modularisation is widely used in higher education; for 
example, in Europe, the UK, Australia, New Zealand and the USA, and 
involves the design of separate, often standalone, units of study. These 
units of study, or modules, can be of varying sizes, can be core modules 
in a degree programme or can be shared between many programmes. 
You may be working in a degree programme structure that is not 
modularised; this chapter provides a useful introduction to modularisa- 
tion, the benefits and pitfalls. 

Modularisation has led to, in some cases, more disparate, less 
cohesive programmes of study. It has also had an impact on assessment 
design, as students can experience a range of diverse assessments and 
have difficulty taking learning from one assignment to the next. The 
Transforming the Experience of Students through Assessment (TESTA) 
project has highlighted these issues. Students need a sense of progression 
in their studies; this chapter offers practical ways to create more cohesive 
programmes. 

The chapter focuses on creating cohesive assessment across a 
programme (and links with Chapter 3, on module design). In particular, 
it will cover: 
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e advantages and disadvantages of modularisation 

e connecting assessments, ensuring that the sequencing of 
assessments allow students to build knowledge, competencies 
and skills 

e reviewing assessment across a programme. 


The Dilemma of Modularisation 


Since the 1990s, higher education institutions in the UK, and other 
countries have adopted modularisation of degree programmes. 
Modularisation entails designing programmes of study in discrete 
units, or modules. In theory, this hugely increases choice for students 
who can select a range of modules and customise their learning. Some 
disciplines, however, have adhered to very tight programmes with a 
limited choice of modules. In some professional programmes, most 
modules are compulsory, so students follow the same programme; for 
example, in law. In other disciplines, core modules provide continuity 
in the programme and a through-line of learning so that each core 
module builds on learning from the previous module; outside of the 
core modules, students have relative freedom to choose from a range 
of option modules. In large interdisciplinary programmes, there may be 
few core modules and an extensive menu of choice. 

The move to modularisation was seen to offer institutions and 
students many advantages, as described in the following (adapted from 
French 2015; Brown and Saunders 1995). 


e Increased choice and flexibility for learners to create their own 
programmes and follow their own interests. 

e Increased opportunities for interdisciplinary programmes, with 
sharing of modules between programmes, thus reducing costs. 
The move to modularisation may have an economic driver as 
modules can be shared across programmes with savings due to 
economies of scale. 

e Greater flexibility for part-time students who do not need to 
follow a set sequence of modules. 

e Marketability of standalone modules; this is of particular 
interest to institutions that want to gain income from providing 
continuing professional development (CPD) courses. 
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e Transfer of credits within the institution and across institu- 
tions, which again provides students with greater choice and 
flexibility. 


However, the disadvantages of modularisation are becoming increas- 
ingly apparent. These are described in the following (adapted from 
French 2015; Brown and Saunders 1995). 


e Organisational issues and keeping track of student learning, 
making sure someone knows about their progress. 

e Choice can be problematic, especially if there is a bewildering 
range of modules to choose from; students may not make the 
best choices and may not understand which modules to pick 
for particular careers, so there is a need for guidance. 

e Mixed levels teaching - teaching students who have no 
previous learning in a subject with those who have. To ensure 
students have the relevant level of knowledge for a module 
they may be required to successfully complete a preparatory 
module (prerequisite). 

e Exponential increases in assessment have been attributed 
to modularisation as each module may require two or more 
assessments (Harland et al. 2015; Jessop and Hughes 2018). 

e Academic ‘fragmentation’ (Brecher 2005) as students don’t 
get a scaffolded understanding of the discipline; this is more 
important in some disciplines, e.g. physics. 


Some degree programmes, particularly in professional fields where 
learning may be regulated by professional bodies, permit very little 
choice of module. These programmes are designed along a more 
traditional linear model with core modules and very few option modules. 
In these programmes, connected through-lines are easier to design, and 
larger modules and synoptic assessments can help to contain the volume 
of assessment (see De Vita 2004 for an evaluation of a synoptic module). 


Modularisation and Assessment 


In a modularised programme, it is important to ensure that assessment 
is connected throughout the programme and that students have an 
opportunity to build on previous learning and develop understanding. 
This can be problematic as some modules may be seen as standalone 
units of learning or may be shared with other disciplinary programmes 
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and be based on very different epistemological beliefs. In these cases, 
the programme leader needs to ensure that students receive adequate 
preparation for all assessment tasks. 

Overassessment in modular programmes has been highlighted by 
Harland et al. (2015). For small, 15 credit modules (in England, Wales 
and Northern Ireland, QAA, 2014) typically two or more pieces of 
assessment are set. Fifteen credits are equivalent to 150 notional hours 
of learning (this includes taught sessions, self-study time and time spent 
on assessments), and to 7.5 European Credit Transfer and Accumulation 
System (ECTS); a bachelor’s degree with honours is typically equivalent 
to 360 credits or 180 ECTS. 

Harland et al. (2015) describe this overassessment as a ‘pedagogy 
of control’ (see also Jessop and Hughes 2018), using assessment to 
motivate and ensure students cover module content. However, an 
increased assessment load can have the opposite effect and result in 
more strategic study practices, with students focusing on the assessment 
tasks with no time for wider reading (Harland et al. 2015). As more 
summative assessment is added to the programme, there is less space 
for formative assessment, so students do not get a chance to practise and 
develop. They may also ignore formative assessments entirely, making 
a strategic decision to focus their efforts on summative assessments. As 
a result, they do not engage in valuable practice activities, which can 
disadvantage marginalised groups. 


Synoptic Assessment 


Possible remedies to the problem of overassessment include synoptic 
assessment (De Vita 2004), where several modules are assessed through 
one assignment. Synoptic assessment combines knowledge and skills 
from two or more modules and allows students to demonstrate their 
understanding of a topic. Synoptic assessments span several modules 
in a programme and require students to synthesise learning from these 
modules, to recognise links between modules and to apply learning to 
real world problems. 

Students can be prepared for synoptic assessments through small 
formative tasks which help them make links between modules and 
synthesise learning. Larger modules can also help to reduce assessment. 
Small 15 credit modules can dramatically push up the volume of 
assessment on a programme, especially if each small module requires 
at least two assessments. Small modules can be replaced with one 
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large 60 credit synthesising module; formative tasks can then prepare 
students for a larger, summative assessment and so decrease the number 
of individual summative assessments. With fewer summative assessment 
tasks, students have more time to tackle formative tasks and more time 
to think and read around the topic. Another benefit is that the teachers’ 
assessment workload is decreased, as formative tasks can be carried out 
in groups and generic feedback can be given on the task. To give generic 
feedback, scan the formative tasks looking for strengths and common 
weaknesses. Provide students with an oral or written summary (either 
online or in class) of what you expected students to achieve, highlight 
some strengths and indicate weaknesses with suggestions for further 
work. 


Transforming the Experience of Students through Assessment 
(TESTA) 


TESTA is a useful tool for exploring assessment practices across a 
programme and identifying areas for development. TESTA was initiated 
by Graham Gibbs, Tansy Jessop and Yassein El Hakim, who designed 
a methodology for collecting data on assessment and feedback in a 
programme. They wanted to address problems of modularisation 
and promote programme assessment practices that enable students 
to understand academic standards on the programme and what they 
need to focus on to achieve programme level aims. The TESTA initiative 
shares concerns about the documentation of standards (Sadler 2014); 
academic standards do not exist in programme documentation, such 
as in learning outcomes and assessment criteria. Students come to 
know these standards through engaging in formative and summative 
assessment tasks and dialogic feedback. 

The TESTA methodology consists of a programme audit, collecting 
data on ‘volumes of summative and formative assessment, varieties of 
assessment, proportions of exams, and the volume of written and oral 
feedback’ (Jessop et al. 2014a, 21). A questionnaire is distributed to 
all final year students on the programme; the ‘Assessment Experience 
Questionnaire (AEQ), is a 28 question survey based on assessment 
principles distilled by Graham Gibbs’ (Gibbs and Simpson, 2005). Finally, 
a focus group is carried out with a selected group of final year students. 
The mix of quantitative and qualitative data provides a good profile of 
assessment on the programme. In the analysis of the data, qualitative 
data is drawn on to illuminate quantitative data and helps to tease out 
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issues such as overassessment, lack of alignment of assessments and the 
quantity and usefulness of feedback. 

In a comparison of data from 23 TESTAs, Jessop et al. (2014b) 
found that some programmes had six times as much assessment as 
others and that final year students were still vague about academic 
standards on their programmes: ‘In spite of well-documented modules 
and programmes, outlining aims, learning outcomes and assessment 
tasks, final-year students were not clear about goals and standards ...’ 
(Jessop et al. 2014b, 82) 


Putting this into Practice: Reviewing an Existing 
Programme 


Degree programmes developed over a period of time may lose focus as 
new content is added and assessments changed. Some large programmes 
can have labyrinthine structures, making links between modules and a 
sense of progression problematic. 

If you are working on an existing programme, start by analysing your 
programme. You can either initiate this as a member of the programme 
team or as a team leader. Use the review questionnaire shown in Table 
4.1 and work with colleagues and students to review assessment across 
the programme, identify connections between assessment tasks and 
through-lines of activities that enable students to practise and perfect 
performance. 

If you are using this questionnaire to plan a new programme start 
by deciding what you want students to learn across the programme and 
understand how each module contributes to that learning (see Chapter 3 
for an explanation of backward design). Think about the complex learning 
that students need to engage in and how the programme will introduce 
and give practice in critical thinking, exposition, synthesis etc. 

Check that the learning on the programme is at a suitable level. 
If comparable national standards are expected, check the appropriate 
documentation, for example, The Framework for Higher Education 
Qualifications (FHEQ) and subject benchmark statements or similar 
national documents (QAA 2014, QAA 2019a). Bear in mind the 
limitations of language in documenting standards (see Chapter 6 for 
a full discussion) and consider whether you are able to judge if the 
programme modules are set at an appropriate level. 
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Table 4.1: Reviewing an existing programme (sources: adapted from 
Bean 2011; University of Suffolk 2019; University of Plymouth 2019) 


Quantity of assessment 


e Is there any evidence of overassessment? Modularisation may mean that 
there are several assessments for each module, leading to overassessment 
and that the same knowledge/skill is repeatedly assessed while others are 
not. You may want to use a tool such as the Transforming the Experience of 
Students through Assessment (TESTA http://testa.ac.uk) research toolkit 
or similar to identify the amount of assessment on the programme, the 
number of varieties and links between assessments. 


Comment: 


Planned development: 


e Are there opportunities to introduce synoptic assessment? Synoptic 
assessments can be more meaningful for learners as they enable students 
to make links between modules. Overall summative assessment is reduced, 
so learners have more time to spend on fewer, synoptic assessments. 


Comment: 


Planned development: 


Through-lines and connections 


e Are there through-lines of assessment? Are there key modules where 
disciplinary understandings/knowledge/skills will be developed? 


Comment: 


Planned development: 


e How do subsequent modules build on this knowledge/these skills? Where 
are the gaps? 


Comment: 


Planned development: 


e How are students prepared for assessments; where do they get practice for 
subsequent assessments? 


Comment: 


Planned development: 
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e How are students involved in assessment design? (e.g. co-construction of 
assessment tasks, assessment criteria, peer review etc. See Chapter 7.) 


Comment: 


Planned development: 


Timing of assessments 


e Are assessments bunched together? This typically happens when 
assessments occur at the end of a module. Change deadlines or reduce 
assessment to avoid bunching, and where bunching can’t be avoided, come 
up with strategies to ease student and staff workload (e.g. preparation time 
for students before exams, marking period afterwards for teachers with no 
other deadlines). 


Comment: 


Planned development: 


e How many varieties of assessment are there in the programme? (e.g. 
lab reports, case studies, MCQs/SBA, OSCEs etc. (see Table 3.1 in 
Chapter 3). Diversity in assessment is good, because it helps develop 
students’ communication skills, enabling communication with different 
audiences using different modes of communication (blogs, presentations, 
formal reports, etc.). But students need to be prepared and have an 
opportunity to practise each new type of assessment; this can be done 
through formative assessment tasks, which help prepare students for 
summative assessments. Formative and summative tasks need to be closely 
linked so students can see the value of completing them. Assessment tasks 
need to be linked into the overall structure of the programme. 


Comment: 


Planned development: 


Inclusive assessment 


e Is assessment planned to support a diverse student cohort? (Does the 
curriculum reflect that diverse cohort, e.g. tasks that give students the 
opportunity to explore their neighbourhoods and indigenous knowledge?) 


Comment: 


Planned development: 
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e Are students given a choice of tasks so that they can choose types that 
allow them to show attainment (see University of Plymouth 2019, and 
Chapter 9)? 


Comment: 


Planned development: 


Academic standards 


e How are students helped to understand assessment standards? (See 
Chapter 7 and discussion of designing activities for preparation, practice 
and partnership.) How will they be able to recognise good quality work? 
What activities will help them gain this understanding? (See Chapter 7.) 


Comment: 


Planned development: 


e How is student progress monitored and supported? And how are they 
supported to monitor and direct their own development? See section on 
ipsative assessment in Chapter 8. 


Comment: 


Planned development: 


e How will students learn about ways of thinking within the discipline and 
how will they be helped to express their thinking in appropriate disciplinary 
discourse (in oral, written, multimedia assignments)? Are writing-to-learn 
(WTL) tasks embedded in lectures, seminars, online activities, fieldwork 
and tutorials? (See Chapter 2 on writing in higher education.) 


Comment: 


Planned development: 


e How will students learn from assessment tasks and show their learning? 
Validity is important here (see Chapter 3). Do the assignments allow 
students to develop and learn across the programme? Are there synoptic 
assessments, providing opportunities to synthesise learning across 
modules? 


Comment: 


Planned development: 
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Think about how learning for the whole programme (knowledge, skills 
etc.) can best be expressed. Analyse the validity of the assessment 
tasks and ensure they provide opportunities for students to learn and 
demonstrate their learning and are connected to the programme aims. 
It can be helpful to work from exemplars of students’ work on your 
programme (or similar programmes) and identify knowledge and skills 
needed to perform well in assessment tasks. 

Think about how each module in the programme contributes to 
the whole and helps students to achieve the academic standards of the 
programme. Find a way to make this linking apparent to students. Help 
students to produce a map that guides them through the programme, 
shows their learning journey, shows how learning is linked and how 
the skills and knowledge and competencies learnt in one module will 
be developed in subsequent modules. Students need to conceptualise 
this and understand how their module choices connect and how one 
assessment prepares them for a subsequent assessment. 

There may be an obvious structure for the modules, and this 
structure may be disciplinary specific. For example, one teacher 
comments: ‘In my own field of physics, the learning structure is naturally 
hierarchical — there’s no point in teaching students Relativity if they 
don’t understand Newton! These hierarchies place definite constraints 
on when you teach what subject’ (James Fryar, 2012). 


Follow-Up 


Reading 


Read more about overassessment in higher education: 


Harland, Tony, Angela McLean, Rob Wass, Ellen Miller and Kwong 
Nui Sim. ‘An Assessment Arms Race and Its Fallout: High-Stakes 
Grading and the Case for Slow Scholarship’, Assessment and 
Evaluation in Higher Education 40, no. 4 (2015): 528-41. 


Investigating Your Practice 


Work with colleagues and students to create a diagram for your 
programme showing connected through-lines of assessment activity. 
Indicate how learning from one assignment can be used in subsequent 
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assignments. Share the diagram with students (e.g. in module 
handbooks) and explain the connections. 

Have discussions with students and colleagues about ways to 
reduce summative assessment on your programme; for example, 
combining modules and developing synoptic assessments, moving or 
splitting assessments to alleviate bunching. If the module assessment is 
six lab reports or six essays, considering changing this to four lab reports 
but with an option for students to submit drafts on the first two reports. 
Student workload is then reduced, they have more time to develop 
the first two reports and they receive feedback on their drafts with 
suggestions on how to develop them. 
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Chapter 5 


Reaching Out: Making Links With 
Communities 


Well-designed research-based assessments can help students to engage 
with the world beyond higher education and contribute critically to policy 
decisions. Higher education institutions are part of the wider community 
and are often funded by and accountable to that community and its 
representatives, for example, political bodies. Higher education institu- 
tions often include in their ‘mission statements’ an aim of reaching out 
and working in partnership with local communities. Links can be created 
and strengthened through assessments involving students working with 
community organisations, such as non-governmental organisations 
(NGOs) and community groups, government departments, and local 
and international employers. Community linked assessments can also 
give students experience of using research to contribute to knowledge 
building within their community and experience of presenting to and 
writing for external audiences. This chapter explores the practicalities 
of setting up assessments which have an external element, liaising with 
external bodies and designing these assessments. Beginning with a 
consideration of the idea of authentic assessment, the chapter explores 
types of inquiry-based learning (IBL) and ways of linking academic 
research with community involvement through imaginative assessment 
tasks which aim to develop students as independent researchers. 


Communicating with Audiences 


Authentic assessment, that is assessment that mirrors practices in 
work and the community is increasingly used in higher education; 
for example, scenario-based assessments which require students to 
respond to professional issues. Scenarios can be designed to mirror 
professional tasks, requiring groups of students to collaborate to solve 
an ill-structured problem (Voss and Post 1988), that is, a problem with 
no correct answer. In solving the task, students combine theoretical 
knowledge of the subject, understandings of how knowledge is 
created and communicated within a discipline (Barradell et al. 2018, 
McCune and Hounsell 2005) with professional skills and so gain a 
better understanding of the application of theory. As problems to be 
investigated are ill-structured, student responses can be divergent. 
Students have an opportunity to explore creative responses to the 
problem. In some cases, this may lead to creation of new knowledge 
in the field or profession (e.g. Chang 2005; Chang and Jackson 2007). 

One step further to is develop assessment in partnership with 
external bodies, ensuring that the assessment task will have diverse 
audiences. This approach to assessment requires careful planning; 
many practical and quality assurance issues need to be thought through. 
Traditionally in higher education, the teacher is the audience for students’ 
assessed work, whether it is a lab report, mathematical calculation, essay 
or presentation. Involving external bodies can be a motivating factor 
for students as they can connect their work in higher education with 
industry, employment and with issues in their own community. 

The assessment for this type of task can take a non-traditional 
form, so rather than essays or lab reports, students may give presenta- 
tions, write blogs, design multi-media training resources, make videos 
and write public information leaflets. Students learn to communicate 
with different audiences. The experience gained from these assessments 
can be valuable when they leave higher education and apply knowledge 
to the world beyond (Boud 2000). Boud and Soler define sustainable 
assessment ‘as a way of rethinking outcomes, curriculum and pedagogy 
away from a focus on disciplinary knowledge to what students can do 
in the world’ (2016, 401). Sustainable assessment, and the notion of 
assessment for learning, position assessment as an integral part of the 
learning experience, rather than assessment of learning. Sustainable 
assessment tasks go beyond memorisation and regurgitation of facts, 
and engage students in complex learning, requiring the application 
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of theoretical knowledge to formulating and solving professional or 
community authentic problems. Learners apply knowledge to community 
or work issues, but this is not about creating ‘worker identities’ (Tennant et 
al. 2010, 107) or producing students ready for employment. Sustainable 
assessment has the wider aim of developing students and enabling 
them to join and better understand their communities, providing them 
with the knowledge and skills to contribute through active citizenship. 
The non-governmental organisation Citizens UK (2019), works with 
the academic community to investigate issues in the local community 
and campaign for action. Academics work in partnership with Citizens 
UK and contribute to their campaigns. Students engage with these 
campaigns and gain a better understanding of their local community and 
democratic processes through carrying out research work. 

Authentic assessment is not unproblematic. Issues can arise when 
scenarios and problem-based learning (PBL) are set up so that there is a 
right answer. This can be reassuring for teachers, who are provided with 
crib sheets and are confident that they have the best possible answer, 
but can create an inauthentic feel. Lack of external involvement can also 
lead to inauthenticity. If there is no external involvement in the scenario, 
e.g. community groups or industry contacts, the assessment can become 
just another exercise communicating with the teacher. Other issues arise 
when enthusiastic teachers design a variety of unfamiliar assessment 
tasks for students; for example, blogs, vlogs, public information leaflets, 
posters and hyperlinked articles. These can be valid tasks but the sheer 
variety and lack of familiarity with these genres can create problems. 
Students may have had no previous experience of making videos or 
writing for the public. A change in the genre of assessment needs careful 
preparation; students need practice in the new genre so that they can 
achieve mastery before being assessed and before their performance is 
summatively judged. Similarly, teacher assessments of these innovative 
tasks may be unreliable, because teachers may not have experience of 
assessing innovative tasks and may not have discussed with colleagues 
how to assess these tasks or agreed academic standards (see Chapter 6). 


Research-Based Learning 
Citizenship in our ‘supercomplex’ world (Barnett 2000) involves 


being able to critique and understand complex issues, recognising the 
limitations of research findings, understanding to what extent research 
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can help towards finding solutions and how best to implement and 
critically evaluate those solutions. 

In higher education, many students are studying for entrance to 
professions such as medicine, law and engineering (Brew 2006); their 
aim is not to become academics. However, the impact and influence of 
academic research and ways of thinking and practising (WTP) in the 
disciplines reaches into all aspects of our professional and personal lives. 
McCune and Hounsell (2005) argue that WTP describe the breadth of 
learning that a student might engage in and that might give students 
insight into the way professionals think and express themselves. For 
students to effectively influence their professions and communities, 
they need to be cognizant of established ways of knowing and of ways 
of creating knowledge within their discipline(s) so that they can engage 
in knowledge building. In developing this awareness, they are also able 
to critically evaluate the established ways of knowing and, through 
critical appraisal, identify shortcomings and dispute current orthodoxy. 
Students may also understand how to use research findings to explore 
and resolve complex challenges, which may be professional, personal 
or societal. This then is the rationale for engaging students in authentic 
research experiences; to provide opportunities for them to develop the 
knowledge and skills they need to function effectively in a supercomplex 
world. 


Views of Research-Based Learning 


Research-based learning epitomises a shift in ways of thinking about 
learning; research-based learning places the student at the centre 
of learning. One of the teacher’s roles entails designing ‘scaffolding’ 
activities (Bean 2011) to help students acquire the skills and knowledge 
needed to plan and conduct research. 

Research-based learning embraces a wide range of practices, e.g. 
inquiry-based learning, object-based learning, case studies or scenario 
based-learning and problem-based learning. Issues raised in the 
literature are the: 


- power relations between students and teacher 

- degree of support/scaffolding required 

- openness of the task set 

- connection to professional practice 

- authenticity of the task 

- range of genres and audience for the final product(s) 
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- focus on knowledge building or building generic research 
skills. 
(Visser-Wijnveen et al. 2016; Smyth et al. 2016; Aditomo et al. 2013; 
Levy and Petrulis 2012; Brew 2006) 


Brew views research-based learning as a ‘shared encounter between 
individual human beings’ (2006, 14). She discusses the power dynamic 
between student and teacher and the danger of the teacher imposing their 
views and opinions on the less experienced ‘submissive’ student. A key 
principle for Brew is that teacher and student should engage in research- 
based learning as equal partners. Chang’s (2005) work illustrates how 
this can be done. His students engage in research in an area that he is not 
expert in and, through the use of an ‘inheritance mechanism’ — a way of 
handing on research data and analysis to the next student cohort, they 
soon become expert. His role is to act as a guide and facilitator, enabling 
students to progress in their research. Chang explains, ‘In the exam I 
include references to other students’ work. So they come to appreciate 
each other as experts’ (Brew 2006, 176). 

Brew’s model of the relationship between research and teaching 
is dynamic and focused on teachers and students interacting as learners 
and ‘knowledge builders’ (Brew 2006, 31), which she terms ‘inclusive 
scholarly knowledge building communities’. Participants are allowed 
to be involved in research activity and there is an equal partnership 
between participants. This can be difficult to achieve as teachers have 
more ‘academic capital’ than students, so it is important to strive towards 
equality. Brew explains that ‘inclusive’ means ‘democracy in the sense in 
which Dewey used the term, where education is about nurturing human 
growth and development’ (Brew 2006, 34). 


Inquiry-Based Learning 


Levy and Petrulis’s (2012) view of ‘inquiry-based learning’ draws on 
Brew’s work. In a small-scale, qualitative study of first year students’ 
experiences of inquiry-based learning, they identified different 
dimensions of experiences from ‘identifying’ at an early stage in the 
programme, where students work on investigations framed by the teacher 
and explore existing knowledge, to ‘authoring’ where students generate 
their questions and can be involved in original research and knowledge 
building. They argue there is evidence for providing more guided forms 
of inquiry-based learning at the start of a programme, while more open 
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‘authoring’ tasks that empower students to develop their own research 
agendas can be used later (Levy and Petrulis 2012, 98). Their model can 
be used for designing research tasks, helping teachers to identify levels 
of support to provide. 

Aditomo et al. (2013) investigated the design of inquiry-based 
learning tasks. They collected data from 224 academics in three 
Australian universities, one from each ‘type [...] “Sandstones” (most 
research intensive), “Gumtrees” and “Unitechs” (least research 
intensive)’ (Aditomo et al. 2013, 1244). They asked the teachers at these 
universities to send examples of their tasks and objectives. They investi- 
gated whether academic discipline or other contextual factors influenced 
tasks types and objectives (Aditomo et al. 2013, 1243). 

They identified a range of task types and distinguished between 
a focus on content and a focus on practice, that is, the kinds of skills 
students need to develop. ‘Use-oriented’ tasks have practical applications 
and aim to develop professional knowledge and skills. They identified 
four types of use-oriented task, namely: 


e roleplaying (2.3 per cent) 

e enactment of practice (28.8 per cent) — provide service to 
clients, e.g. in medicine (clinical consultations), education, 
accounting etc. 

e applied research similar to simplified research but with a 
practical application (9.3 per cent) 

e simulated applied research (8.8 per cent). 

(N.B.: figures in brackets relate to the percentage of tasks.) 


The remaining tasks were categorised as non use-oriented. Non use- 
oriented tasks were designed to develop knowledge of content and 
categorised as: 


e discussion-based inquiry involving online or face-to-face 
discussion (5.6 per cent) 

e  literature-based inquiry (6 per cent) involving a review of the 
literature with no data collection 

e simplified research with pre-specified research questions and 
methods/analysis provided (27.4 per cent) 

e scholarly research (10.7 per cent), requiring formulation of 
questions and collection and analysis of data. 
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Applied research, simulated applied research and simplified research 
tasks focused on ‘the use and acquisition of disciplinary concepts’ and 
guided students to use particular theories (Aditomo et al. 2013, 1250). 
Scholarly research typically required students to communicate findings 
to external audiences, at least external to the class. 


Examples of Inquiry-Based Learning Tasks 


Here are examples of some inquiry-based learning task types (all 
from Aditomo et al. 2013); no difference was found between types of 
university and type of task set. 


Scholarly Research 


The students are to obtain a data set, ask some statistically based 
questions (their choice) about the data and use statistical methods 
to answer their questions. The students are to summarise their 
work in a short report. Students may work in teams of up to three 
students. (Undergraduate, statistics, 210 students) (Aditomo et al. 
2013, 1246) 


Simplified Research 


Students are given three sets of DNA sequences and need to use 
the computational tools and databases they have learnt about to 
discover the respective functions of the proteins. They are not told 
which tools to use, and the third of the sequences requires some 
detective work (I do give a hint). (Undergraduate, biomedicine) 
(Aditomo et al. 2013, 1246) 


Literature-Based Inquiry 
‘Asked to undertake a systematic review on a question of their 
choice.’ (Undergraduate, psychology, 39 students) (Aditomo et al. 
2013, 1246) 

Applied Research 
[Students] had to research the issues surrounding transport, 


energy, food and waste in residential dwellings in NSW and then 
work out how they could deliver a ‘car free household’, a ‘zero waste 


70 ASSESSMENT AND FEEDBACK IN HIGHER EDUCATION 


household’ or a ‘food producing household’. They worked in teams 
following a process of research, synthesis and communication. 
When they had come up with a range of design solutions, they 
had to pitch these ideas as if to a funding body, seeking start-up 
funding for further development. (Undergraduate, engineering, 
580 students) (Aditomo et al. 2013, 1247) 


Simulated Applied Research 


Students are given several case scenarios related to professional 
practice, ethical behaviour and clinical reasoning during 
practicums. Students discuss these case scenarios in small groups 
and use a problem solving framework to plan appropriate action 
or intervention related to the given scenario. Students feedback 
and discuss their findings and plans for intervention with the 
large group. (Undergraduate, occupational therapy, 75 students) 
(Aditomo et al. 2013, 1248) 


Role Playing 


After the two hour lecture we held a workshop for an hour where 
the class of 35 was divided into groups of three and each group 
was given six scenarios to work through. The scenarios were 
worked through one at a time with one student being the patient 
presenting a prescription, another being the pharmacist dispensing 
the prescription and the third student observing. Then the next 
student would have the opportunity of being the ‘pharmacist’ and 
so on until they had each been the pharmacist twice, the patient 
twice and observed twice. (Postgraduate, pharmacy, 35 students) 
(Aditomo et al. 2013, 1248) 


From this data, Aditomo et al. (2013), developed a typology of IBL tasks. 
Many of the inquiry-based learning tasks collected in this study fell 
into Levy and Petrulis’ (2012) category of identifying, that is, exploring 
existing knowledge. Scholarly research tasks, which account for 10.7 
per cent of the tasks, were the closest to authoring tasks, involving 
knowledge building. The authors point out that concerns about inquiry- 
based learning activities lacking authenticity seem valid, particularly 
when the task is not open-ended but has a right answer. 
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Object-Based Learning (OBL) 


One type of inquiry-based learning task that requires special mention is 
object-based learning, pioneered by university museums and collections 
(Chatterjee and Hannan 2016; Hannan et al. 2013; Duhs 2010; 
Kador et al. 2018; Morgan 2018). Object-based learning can offer the 
opportunity for knowledge building and scholarly research. It is not a 
new pedagogy; objects in museum collections were extensively used in 
teaching before the twentieth century (Kador et al. 2018, 159). Objects 
can be from curated collections or from elsewhere and can be used to 
enhance learning in any discipline. It is ‘a pedagogy that prioritises 
facilitated interaction with “material culture” to enhance critical thinking 
and key skills’ (Kador et al. 2018, 158). ‘Material culture’ is a broad term 
that includes everyday objects. 

Chatterjee and Hannan (2016) emphasise the importance of using 
the senses, especially touch to make sense of an object. Looking closely 
at objects and touching them can bring to life theoretical concepts 
and increase student understanding (Kador et al. 2018). The mystery 
specimen activity (see below) engages students with a mystery object. 
They use all their senses, especially touch, sight and smell, to explore the 
origin and use of the object. Object-based learning can be an opportunity 
for both identifying inquiry-based learning and, in the case of more 
complex investigations, authoring inquiry-based learning as students 
are involved in knowledge building and constructing the histories of 
unknown objects. 


Case Study: Mystery Specimen 


Level — third and fourth year undergraduate (but also used in master’s 
programmes and in a range of modules) 

Discipline — biology and zoology, but could be adapted to many 
disciplines (see below) 

Module — used in a range of modules, e.g. Vertebrate Life and Evolution 
in the Department of Genetics Evolution and Environment, UCL 

Cohort — a wide range, most from biological sciences but also some 
from geography, anthropology or human sciences. Both home and inter- 
national students 

Learning approach — object-based learning, active learning 
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The ‘mystery specimen’ involves students in activities that develop discip- 
linary thinking and mirror disciplinary research. Students work in the 
Grant Museum of Zoology, interacting with museum staff who facilitate 
parts of the module. They are each given an unidentified vertebrate 
specimen and, over a term (a three-month period), they are required to 
identify the specimen. 

The specimens all have museum labels, descriptions and classifica- 
tions removed and can be anything from bone to skin. The student’s task 
is to identify which part of the animal the specimen comes from. They 
must identify the ‘correct class, order, family, genus or species’ (Kador 
et al. 2018, 163). Most students manage to identify which group of 
vertebrate the specimen comes from (e.g. bird, fish etc.). Their research 
helps them identify: 


e the material of the specimen 
e whether the specimen is complete or partial 
e what part of the animal the specimen comes from. 


It can be more difficult for students to identify: 


e whether the specimen is from a male or female animal 
e the age of the animal (adult or juvenile) 

e whether any part of the specimen has been altered 

e any sign of pathology in the specimen. 


Students are required to engage with disciplinary literature and to 
develop disciplinary thinking and research skills. It is not possible to 
identify the specimen from internet searches. Students are supported 
in their learning by a series of teaching sessions. They learn about ways 
of looking at specimens; this involves detailed anatomical observations 
and consulting drawings and photographs. They interact with museum 
staff, for example, they can ask for additional comparable material to 
help them identify their specimen. They can explore other museums and 
consult relevant literature. In this way, they are developing the research 
skills required for specimen-based research. 

Assessment of the module draws on the research skills that 
students have acquired and involves writing an article for the scientific 
journal, Trends in Ecology and Evolution, following the journal’s author 
guidelines. The assessment gives students practice in communicating 
with an external audience using appropriate conventions. In the article, 
students are expected to propose further research work that would need 
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to be carried out to confirm their identification and reflect on which 
scientific techniques would give a more accurate identification, for 
example DNA sampling or micro-CT scanning. The articles are assessed 
on the quality of the research work, not on whether the identification of 
the vertebrate is correct. 

This work also benefits the Grant Museum as student identifica- 
tions are compared with descriptions in the Museum’s catalogue and, 
in some instances, this has led to discoveries that specimens have been 
misidentified. These misidentified specimens are now on display in the 
museum, celebrating the students’ detective work. 

Students’ perceptions of object-based learning was investigated in 
a survey of 154 students from a range of disciplines; the authors reported 
that 61 per cent of students agreed that object-based learning is more 
effective than listening to a lecture and valued object-based learning 
because it is visual, exciting, has practical application, and is good for 
developing teamwork, communication and observation skills (Hannan 
et al. 2013). 


Preparation 


This activity is an example of an authoring activity and, as such, assumes 
considerable knowledge of research skills and academic writing. 
Students will need to have completed activities previously in order 
to be familiar with the research skills and knowledge needed for this 
kind of object-based learning. Identifying activities such as ‘Meet the 
Researcher’ (see Chapter 2) will introduce them to research areas and 
literature. In addition to knowledge and research skills, they will need 
practice in journal article writing; this is a high-level skill which will need 
to be developed throughout the programme of study. (See Chapter 4 for 
a discussion on planning assessment across a programme.) 


Adaptation: Use in Other Disciplines/at Other Levels 
The activity could be simplified for first year students by asking students 
to work in groups and blog about their mystery specimen. Other objects 


such as artwork and historical artefacts can be used for object-based 
learning in a range of disciplines. 
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Further Reading 


For a more detailed explanation of the mystery specimen, and more case 
studies of object-based learning, see the two titles below. 

Kador, Thomas, Leonie Hannan, Julianne Nyhan, Melissa Terras, 
Helen J. Chatterjee and Mark Carnall. ‘Object-Based Learning and 
Research-Based Education: Case Studies from the UCL Curricula’. In 
Teaching and Learning in Higher Education: Perspectives from UCL, edited 
by Jason P. Davies and Norbert Pachler, 157-76. London: UCL Institute 
of Education Press, 2018. 

Hannan, Leonie, Rosalind Duhs and Helen Chatterjee. ‘Object- 
Based Learning: A Powerful Pedagogy for Higher Education’. In Museums 
and Higher Education Working Together: Challenges and Opportunities, 
edited by Anne Boddington, Jos Boys and Catherine Speight, 159-68. 
Farnham: Ashgate Publishing, 2013. 


Disciplinary Knowledge 


Ashwin et al.’s (2017) study of undergraduate dissertations, highlights 
what students gain from assessments that focus on disciplinary 
knowledge (producing and authoring in Levy and Petrulis’ 2012 terms) 
rather than generic research skills. They conclude: 


... it seems that students’ engagement with disciplinary knowledge 
through research is a key mechanism for the subject-based 
benefits that are provided through research-based learning. The 
implications of this are that these benefits are not likely to be 
derived from students doing any form of research. Unless they see 
their disciplinary ways of thinking and practising (McCune and 
Entwistle 2011; McCune and Hounsell 2005) as providing a way of 
answering their questions, engagement in inquiry-based learning is 
unlikely to lead to changes in students’ understanding of academic 
knowledge. (Ashwin et al. 2017, 528) 


Advantages of Non-Traditional Assessments for Students 
Traditional forms of assessments in higher education, such as essays and 


multiple-choice questions, which do not involve students in research- 
based learning or link to an external context are limited. The advantages 


REACHING OUT 


75 


of the non-traditional assessments described in this chapter are described 
here (adapted from UNSW 2019). 


- Learning is linked to a context beyond the higher education 
institution. In this context, students have the opportunity to 
apply knowledge and skills or develop new knowledge or skills. 

- Assessment can be more holistic and can help students bring 
together learning from a range of modules and link theoretical 
knowledge and professional knowledge. 

- | Assessments can align more with the kind of activity students 
would do outside of the higher education institution, either 
as employees or citizens. Students can see the benefit of the 
assessment; it appears more valid. 

- Research-based learning that focuses on knowledge building 
can shift the power balance in learning and assessment (Brew 
2006). Students may explore areas that are unfamiliar to 
teachers. Students can become ‘experts’ in those areas (see 
Chang 2005). 

- Research-based learning can enhance employability; employers 
like to know that students can apply knowledge, work in teams 
and communicate with the public. 


Advantages of Research-Based Learning for Teachers 


Teachers can benefit from research-based learning in the following ways 
(adapted from UNSW 2019). 


- Teachers often comment that students do not make links 
between modules. Research-based learning assessments can 
help students make links between modules so that they develop 
a more holistic view of their programme. 

- Research-based learning can keep teachers up-to-date with 
industry, professional and community developments and 
practices. Students may explore new areas and add to areas 
studied, helping teachers to expand their knowledge; in Levy 
and Petrulis’ terms (2012) students are engaged in ‘authoring’ 
activities. Teachers may come to see students as experts and 
facilitate publication of their work (Chang 2005). 

- Research-based learning can help students develop as 
independent learners. 
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Disadvantages and Challenges of Research-Based Learning 


The disadvantages of research-based learning (adapted from UNSW 
2019) are as follows. 


- Unfamiliar genres and tasks. Some students may be disad- 
vantaged if research-based learning assessments require 
unfamiliar genres, e.g. a video, case study report, maths lesson 
plan and resources, public information leaflet or academic 
article/book chapter. If students are not prepared, they can 
flounder with new ways of studying and unfamiliar genres. It is 
important to plan preparation activities and give students time 
to practise new genres. Build in lots of identifying activities 
(Levy and Petrulis 2012) at the start of the programme. 

- Time commitment. Research-based learning can take much 
longer than traditional assessments and there is a limit to what 
can be explored in a short module or even one year. Chang’s 
(2005) inheritance mechanism (see case study below) enables 
handover of work to the next cohort, so that students can 
quickly pick up and develop their peers’ work. 

- Accessibility. If students are required to visit external sites, 
these need to be fully accessible and safe environments. Timing 
of visits should respect religious holidays (see Chapter 9 for 
more on inclusive assessments). 


With these considerations in mind, teachers need to think about the 
increased time commitment needed to develop and plan links with 
community groups or health care providers or industry. Wills (2014) 
suggests that the higher education institution should have pre-existing 
links with community organisations and should be contributing to work 
in their communities, ‘developing long lasting, reciprocal relationships’. 
She comments on the value of fieldwork for geography students: ‘In 
our experience at Queen Mary, geographical field study is a vehicle for 
learning about the ways in which local institutions and their people work 
together, the power of relationships to effect political change and the 
dynamics of running campaigns’ (Wills 2014, 281). 

The value of fieldwork in the community is that students learn a 
wider range of skills. Wills (2014) worked with Citizens UK (formerly 
known as London Citizens). Her students carried out research in a multi- 
cultural inner-city area with an non-governmental organisation that is 
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working ‘across divisions of faith, ethnicity, gender and age through a 
focus on working together for the common good of the city’ (Wills 2014, 
281). Students developed a shared project with Citizens UK members 
and learnt, ‘... a variety of other skills such as, in our case, research 
design, planning and methodology, team working, time management, 
presentation and professionalism’ (Wills 2014, 281). Moreover, as some 
home students lived in the multicultural inner city, they could bring their 
knowledge and understanding of community issues to their research. 
Interestingly, this local knowledge can shift the power dynamic between 
students and teacher; teachers may have more ‘academic capital’ (Brew 
2006, 31) than students, but students may be more cognisant of local 
issues and can use this knowledge in interpreting research data. 

In creating and maintaining links with external partners, it is 
important to delineate a clear role for them. The role of the external 
partner in the assessment process needs to be specified and in line with 
institutional policies and regulations. Other considerations, discussed 
in subsequent chapters, are grading of new forms of assessment; for 
example, video assignments and assignments that include web design. 
Teachers need to think through what degree of expertise in video etc. is 
expected. If students are involved in designing assessments or involved 
in peer assessment, they need to be competent to carry out these roles. 
Chapter 7 discusses ways of preparing students for peer and collabora- 
tive assessments. 


Overcoming Challenges 


The challenges listed above can be overcome through careful preparation 
of both students and external partners. The roles of all parties need to be 
clearly defined. In research-based learning, the teacher’s role changes, 
and s/he becomes a mentor, guiding students in their discovery, advising 
on time scales and research methodology. The teacher ensures groups do 
not take on too much and are realistic about what they can achieve. The 
teacher also ensures that external partners have realistic expectations of 
student outputs. As with all assessments, it is important to plan research- 
based learning activities across the programme (see Chapter 4 on 
programme design). The assessment may be synoptic, requiring students 
to draw on learning from several modules and address concepts central 
to the discipline(s); for example, a final year dissertation project. 
Teachers should ensure that students have practice in ‘bite size’ bits 
of the research-based learning assessment; for example if the assessment 
involves collecting data and writing a briefing report for an external 
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audience, ensure that students have a task in the first term of the first 
year that involves data collection in groups, and in the second term, 
build on this with a task that requires more data collection and analysis. 
Students should be required to write a short version of a briefing report 
and present to an external audience, so that they have practice in the 
genre and content before they tackle a summatively assessed briefing 
report. When research-based learning assessments are set, opportunities 
for discussion of early plans (e.g. group presentations) and feedback on 
drafts should be provided. 

External partners can be an invaluable source of information. 
Involving them in the design of the assessment can help ensure authenti- 
city, for example by developing a task that mirrors professional practices. 
Having an external audience, someone who is steeped in the profes- 
sional community of practice, can add an extra dimension to student 
learning and help focus the task. For students, additional learning may 
occur as they interact with members of the external community. For the 
teacher, much time may be spent in making contact, designing a valid 
assessment task and negotiating external involvement in line with insti- 
tutional assessment policies; check the quality assurance guidelines for 
your institution. 


Case Study: Authoring in the Physical Sciences 


Module title — topics in the history of the physical sciences 

Level — undergraduate 

Cohort — home and international students 

Learning approach - research-based learning, ‘directed community 
model’ (Chang 2005, 389) 

What’s of interest — students develop as experts and cite each other’s 
work, pass work on to incoming students through an ‘inheritance 
mechanism’ (Chang 2005, 391) and publish collaborative work. 


Chang’s way of working with students is fascinating. He has created a 
‘directed community’ of student researchers, engaged in knowledge 
building and creating original work that has been published in a collected 
book. Student assessment has become a learning opportunity for Chang 
himself, as his students research areas in which he is not an expert. His 
role is to guide and facilitate student research. 

Chang felt frustrated that the excellent quality of work his students 
were producing could not be published and thus reach a wider audience. 
A single module did not allow students enough time to create publishable 
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work that could be seen as contributing to research and building 
knowledge. Student research projects needed to be developed over a 
longer period of time, so he created the idea of a ‘directed community’ 
of students working together and sharing work through an ‘inheritance 
mechanism’ (Chang 2005, 391). 

Students work on a theme that is flexible enough to accommodate 
groups of students over a period of years; the chosen theme was ‘history 
of the chemical element chlorine’ (Chang 2005, 388). Each student 
has an individual project but all projects are connected to the common 
theme of the history of chlorine. Students carefully document their work, 
creating records that are passed on to student cohorts in subsequent 
years. Over a period of four years, the data collected and work done by 
students contributes to knowledge building in the field. 

Chang describes his approach as a ‘directed community model’ 
of research where students are involved in working independently on 
their own research projects, supported by the teacher and their peers. 
The teacher introduces them to ways of thinking and practising in the 
discipline (Entwistle 2005), and students are guided to make contact 
with experts in their topics. Use of their predecessors’ work quickly 
introduces them to ways of working and they learn to cite each other’s 
work and refine and develop their predecessors’ work. This ‘inheritance 
mechanism’ (Chang 2005, 391), inheriting work from previous students, 
allows students to make further progress with the research and reach 
a publishable standard. Students choose whether to make their work 
available to the next cohort; only one student had opted not to do this. 
In the published book (Chang and Jackson 2007), all students who have 
contributed to the work are listed as authors. 


Assessment 


The module is assessed through an essay and exam. Students submit 
with their essay all their research records, e.g. annotated bibliography, 
reading notes, literature searches, photocopies of materials, drafts, work 
plans and correspondence (Chang 2005, 391). Inheriting this wealth of 
material enables students to build on previous work and also to learn 
how to carry out research. In addition to the essay, students take an exam 
answering questions on their peers’ work and writing an essay about the 
research process. Chang (2005) points out that students in this module 
achieve higher grades compared to the other modules they take on the 
programme. He has received enthusiastic reviews from students about 
their experiences of learning on this module. 
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Follow-Up 


Reading 


For an introduction to research-based learning, issues and challenges, 
read Angela Brew’s Research and Teaching: Beyond the Divide. Basingstoke: 
Palgrave Macmillan, 2006. 

For an engaging account of one teacher’s experience of introducing 
research-based learning, read Hasok Chang’s ‘Turning an Undergraduate 
Class into a Professional Research Community’, Teaching in Higher 
Education 10, no. 3 (2005): 387-94. 


Investigating Your Practice 


Use the advice in the UNSW (2019) website to design a new research- 
based learning activity for your programme or to review an existing 
activity. Work with students and colleagues to identify issues and 
areas for change. If there is no ‘inheritance mechanism’ (Chang 2005) 
consider whether it would be beneficial to introduce one. An inheritance 
mechanism can ensure that students are involved in knowledge building 
in the discipline and work as a community to develop an area of research 
over a period of years. 

Consider ways of introducing students to local democracy within 
the research-based learning activity, working on shared research with 
community groups. This can introduce students to democratic politics, a 
much wider demographic and to local activism, teaching them skills for 
citizenship (see Wills 2014). 
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Part 2 
MAKING JUDGEMENTS 


Chapter 6 
Marker Reliability 


Marker reliability in higher education is the elephant in the room. 
Academic staff experience considerable anxiety around marking 
and frequently experience disagreements and inconsistencies when 
marking with colleagues. A large number of studies have found that 
marker judgements in controlled conditions are unreliable. This chapter 
explores academic standards, drawing on a project aimed at developing 
comparable, national academic standards. I will explore the complexity 
of assessment judgements in higher education, dig further into the 
limitations of practices that claim to develop more reliable marking (e.g. 
assessment criteria, harmonisation) and propose practical solutions to 
enable more reliable assessment judgements. 

Assessment is highly regulated, both within institutions and 
externally. In the UK, the Quality Assurance Agency (QAA 2019b) offers 
guidance on quality assurance through, for example, the Quality Code 
which covers threshold levels, academic standards and academic quality. 
Institutions are expected to provide details of assessment standards; for 
example, module and programme assessment criteria, rubrics and level 
descriptors. Despite these explicit descriptions of standards, marking 
in higher education is found to be unreliable (Bloxham et al. 2016). 
Students recognise that teachers’ judgements of the quality of their 
assignments vary. Typically, they describe markers as ‘hard’ or ‘soft’ 
markers (O’Donovan 2019). 
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Why are Marking Judgements Unreliable? 


The unreliability of marker judgements has been a feature of many studies 
on marking in higher education (e.g. Williams and Kemp 2019; Bloxham 
et al. 2016; Bloxham et al. 2015; Tisi et al. 2013). Notwithstanding this 
research, markers will often claim that they mark similarly to colleagues 
in their programme, but there may be reasons for this. 


- Teacher knowledge of the student cohort and expected grade; 
there may be an expectation that most students will get a good 
degree, so markers use a very narrow band. 

- Power relations; teachers follow the lead of the marker with the 
highest status within the department, e.g. a senior professor. 


In addition, the assessment load on teachers, when they mark and where 
they mark can all have an impact on markers’ judgements. In the research 
literature, suggested reasons for variation in marker judgements are as 
follows. 


- Seeing another marker’s mark. If second marking is not ‘blind’ 
then the second marker will be influenced by the first marker’s 
grades. 

- The type of assessment task - ill-structured tasks (Voss and 
Post 1988) produce more variation because there is no agreed 
correct answer. In higher education most tasks that aim to 
assess complex learning are open-ended with no one correct 
answer. There is considerable subjectivity in judging the 
quality of students’ work, especially judgements on creativity, 
quality of argument and criticality. 

- The design of the assessment task may also produce variation. 
Innovative assessments such as blogs, vlogs, video and 
web-based assessment may be judged differently as assessors 
may, irrespective of assessment criteria, be influenced by the 
form the assessment is presented in, rather than the content, 
e.g. quality of video production. 

- Lack of harmonisation (also called pre-moderation or parity 
meetings) of assessment standards. Markers do not meet before 
grading to discuss exemplars of students’ work and agree 
standards through discussion, especially focusing on levels. In 
a harmonisation discussion, markers can consider what they 
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value in the exemplars and what constitutes different levels of 
work, e.g. good quality work, a borderline pass etc. 

- Marker backgrounds may cause variation. Markers may have 
different disciplinary backgrounds and different epistemo- 
logical understandings (Lea and Street 1998). Tacit knowledge 
(Polanyi 2009) influences marker’s judgement (Sadler 2014; 
Hunter and Docherty 2011) and individuals have different 
influences on their tacit knowledge. 

- Assessment criteria is not comprehensive. The range of 
influences which markers draw on in making a judgement 
cannot be represented in one small list of pre-set criteria 
(Sadler 2014). Typically, markers add other criteria, ignore 
some criteria or have different interpretations of criteria 
(Bloxham et al. 2016). 

- Assessment criteria and other documents that try to ‘describe’ 
standards are limited, e.g. Framework for Higher Education 
Qualifications (QAA 2014) and Subject Benchmark Statements 
(QAA 2019a). The language used in documentation is open to 
a range of interpretation. 


Other factors that can affect markers’ judgements are: 


- the time of day 

- marking a good assignment after a weak one 

- student trajectory — if marking is not anonymised, markers can 
be influenced by their knowledge of the student and whether 
the current assignment is better/worse than the last one. 


Reducing Variation Through Documentation of Standards 


Attempts to reduce variation in marking judgements centre around 
descriptions of standards in written documents, such as level descriptors 
and assessment criteria, which aim to describe academic standards in a 
specific context (e.g. in a module, programme, discipline, professional 
qualification). Examples of such documents are the Framework for Higher 
Education Qualifications (QAA 2014), Subject Benchmark Statements 
(QAA 2019a) and professional and regulatory bodies’ standards, e.g. 
the General Dental Council (2019). Making standards explicit involves 
describing the standard in language; the assumption is that this will 
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enable assessors to make comparable and reliable judgements about the 
quality of student assignments. 

These reliable judgements would then be consistent across time and, 
in countries where national standards are required, across programmes 
and institutions. In these countries, there is an expectation that higher 
education institutions’ degree awards conform to a shared disciplin- 
ary standard and that there is an understanding across disciplines of 
academic standards, so for example the UK Quality Code states that: 


The value of qualifications awarded to students at the point 
of qualification and over time is in line with sector-recognised 
standards. [...] 

Higher education providers are expected to ensure that their 
standards are consistent with documented standards [...] 

The academic standards of courses meet the requirements of 
the relevant national qualifications framework. [...] 

And there is an expectation that assessment of student 
performance will be “reliable” [...] 

Courses are well-designed, provide a high-quality academic 
experience for all students and enable a student’s achievement to 
be reliably assessed. (QAA, Quality Code 2014) 


The assumption in the Quality Code in the UK is that there are ‘sector- 
recognised standards’, that these standards will remain the same ‘over 
time’ and that an institution’s standards are ‘consistent with the relevant 
national qualifications frameworks’. The mechanism for ensuring 
sector-wide comparable standards is a reference to national frameworks 
and ‘external expertise’ so that: ‘Degree-awarding bodies engage 
independent external examiners to comment impartially and informa- 
tively on academic standards, student achievement and assessment 
processes for all provision that leads to the award of credit or a qualifica- 
tion’ (QAA 2019c). 

In other countries, institutions may be autonomous, and so may 
expect only that marking within a programme in the institution should 
be consistent. They may expect that the programme team should have a 
shared understanding of good quality work at undergraduate and post- 
graduate levels and that they apply this standard across time, so that the 
current degree standard is the same as past awards. 

In Australia, there has been a shift to establishing national 
standards. For example, Watty et al. report that: 
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In accounting education in Australia, these standards are now 
determined at the national level and are disciplinary focused, 
rather than at a local/university level which has been the practice 
in the past. Higher education institutions are still free to expand 
and build on these minimum standards to differentiate their 
programmes (courses of study). (2014, 461-2) 


The mechanisms for ensuring national qualification standards in 
Australia have not been specified: 


The emergence of these standards has, in turn, given rise to a 
question as to how higher education institutions are to provide 
evidence that their students have achieved the standards on 
graduation. At the time of writing, no formal process has been 
nationally accepted in Australia for providing this evidence. (Watty 
et al. 2014, 462) 


In the UK, degree standards are assumed to be upheld through external 
peer review. However, the process of external examining has come 
under scrutiny (Bloxham and Price 2015; Medland 2015). An external 
examiner is a subject specialist appointed by an institution to peer 
review the module(s) and to monitor assessment design and assessment 
processes with the aim of assuring quality and comparability of academic 
standards. Universities UK explains the British system of external 
examiners: 


All UK universities have long made use of a network of independent 
and impartial academic advisers, called external examiners. 
These are drawn from other institutions, or from areas of relevant 
professional practice. External examiners report to the Vice- 
Chancellor of the university on whether the standards set are 
appropriate, by referring both to their experience of standards in 
other universities, and to the Academic Infrastructure established 
by the Quality Assurance Agency (the Code of Practice, Subject 
Benchmark Statements, the Frameworks for Higher Education 
Qualifications, and institutional Programme Specifications). 
The aim is to ensure that the threshold standards of student 
performance are comparable with those of students following 
similar courses in other UK universities. External examiners 
provide authoritative advice on the extent to which the processes 
for assessment, examination and the determination of awards are 
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sound and have been applied fairly. External examiner reports have 
significant status within the university. They are directed to the 
Vice-Chancellor and are considered at, and used by, the department 
and university in internal quality assurance committees. (2008, 5) 


The role of external examiner is independent of the institution; the 
external examiner is expected to follow the institution’s policies and 
procedures and is paid by the institution. The examiner’s role however 
is to offer a critical, peer review of the module, including the syllabus 
and assessment arrangements. S/he is expected to be autonomous and 
objective. Other countries have used an external examining system, e.g. 
Denmark, Ireland, New Zealand, Malaysia, Brunei, India. 

Research on the assessment literacy of external examiners has 
raised concerns about whether external examiners, appointed as subject 
experts have the necessary assessment literacy to make robust decisions 
about comparable academic standards (Medland 2015; Bloxham and 
Price 2015; Bloxham et al. 2015). Following a critical review of external 
examining in the UK and concerns about the robustness of degree 
standards, a government-funded project has been designed to explore 
degree standards in the UK. The purpose of this five-year project is to: 


e design, pilot and deliver different approaches to the profes- 
sional development of external examiners 

e propose evidence-based and cost-effective longer-term 
approaches to the professional development of external 
examiners operating across the higher education system in 
England, Northern Ireland and Wales 

e explore approaches to the calibration of standards, presenting 
recommendations for future work in this area. (quoted from 
Advance HE 2019b) 


Professional development of external examiners is important because 


external examiners may be the only external process to comment on 
academic standards. 


Critique of Codifications 
Written documentation of standards, such as the Framework of Higher 


Education Qualifications and subject benchmark statements aim to 
promote and maintain consistent standards across a discipline or a sector. 
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However, written descriptions of standards are notoriously difficult to 
interpret; the language used in criteria such as ‘argument’, ‘structure’ 
and ‘criticality can be interpreted differently by different markers. 

Sadler (2014) defines ‘standards’ as ‘fixed reference levels of 
attainment’ (2014, 193), as ‘the recognised measure of what is adequate 
for some purpose, so established by authority, custom, or consensus’ 
(2014, 189). Sadler terms all explicit descriptions of assessment 
standards (e.g. assessment criteria, grade descriptors, marking 
rubrics, intended learning outcomes, subject benchmark statements 
and graduate attributes) as ‘codifications’ — ‘Codification is the act of 
developing explicit descriptions of standards...’ (Sadler 2014, 275). 
The aim of codifications is to describe standards so that assessors can 
make similar judgements about the quality of student work. However, 
the descriptions of these levels of attainment in written documentation 
cannot fix standards. Research studies suggest that assessors’ interpret- 
ations of codifications vary (Bloxham et al. 2011). One reason suggested 
for this variation is that the language of codifications is open to interpret- 
ation and because markers will draw on tacit knowledge to interpret, 
they will inevitably come to different understandings. Moreover, codi- 
fications, such as assessment criteria, cannot encapsulate the vast range 
of tacit knowledge that assessors draw on when making a judgement 
(Sadler 2014). 

Sadler (2014) offers a detailed analysis of the limitations of 
assessment criteria, arguing that: 


e there is often overlapping criteria (2014, 276) 

e some criteria are included, others not (2014, 277) 

e there is extensive use of hedge words, e.g. ‘excellent’, 
‘adequate’, ‘mostly’ (2014, 279) which are open to interpret- 
ation by markers 

e more detailed descriptions of criteria do not clarify; they are 
also open to interpretation (2014, 279). 


The ‘fuzziness and elasticity of specifiers’ (Sadler 2014, 286) in 
assessment in higher education results in fuzzy standards. But it is 
impossible to precisely describe these standards; more and more 
description is not going to lead to more exact standards. One reason 
for this is the nature of learning in higher education. The goal of some 
degrees, particularly at postgraduate level, may be to lead learners 
to construct new knowledge (see Chapter 5 for examples of student 
involvement in knowledge building). Criticality, knowledge construction 
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and creativity may be highly valued by teachers but are difficult to pin 
down in codifications (Sadler 2014, 278). 


Ways Forward: How Disciplinary Communities Establish 
Standards 


If standards cannot be reliably described in written documentation, 
how can they be fixed? Butler Shay’s (2004) interesting work on the 
nature of academic standards suggests that standards are held within 
disciplinary communities and are socially constructed within the 
community. Socially constructed academic standards imply that within 
each discipline, teachers can reach agreement on the appropriate 
standard of work at each level of study. Through discussion, standards 
come to be socially constructed within the community; the community 
determines and maintains the standards. In Chapter 2, I discussed the 
idea of learning within a community of practice (Wenger 1999) and the 
way in which knowledge is constructed and held within the community. 
Sadler argues for standards as ‘fixed reference levels’ (2014, 283) and 
‘comparability across both contexts and time requires that standards are 
not only held in common, but also kept secure...’ (2014, 284). He also 
states that ‘if standards could be fixed over time, students’ achievement 
could be graded with integrity’ (Sadler 2014, 283). If we accept Sadler’s 
argument, then there is a need to establish a process whereby disciplin- 
ary communities can discuss and agree standards. In some countries, the 
commentator on standards is the external examiner but, as discussed 
above, studies have shown external examiners are not necessarily experts 
in assessment or national standards so, currently, external examining 
is not a robust process for ensuring standards. An alternative process 
for ensuring comparability of academic standards has been trialled in 
Australia. The use of calibration as a means of establishing disciplinary 
standards has been introduced, and calibration trials have also been 
conducted in the UK as part of the Degree Standards Project (Advance 
HE 2019b). 


What is Calibration? 


Calibration is a process whereby a community of teachers and practi- 
tioners discuss and compare exemplars of students’ work in order to 
reach consensus and agreement on a standard. Calibration has been 
used to agree degree standards, especially borderline classifications, 


ASSESSMENT AND FEEDBACK IN HIGHER EDUCATION 


and understandings of key assessment criteria. Research in this area is 
recent, so there is no body of evidence to draw on to make claims about 
the effectiveness of calibration, but there are studies that suggest that 
calibration activities can lead to more consistent assessment judgements 
(Watty et al. 2014; O’Connell et al. 2016). 

The following case study describes how calibration works. 


Case Study: Calibration of Threshold Standard for Bachelor 
Graduates in Accounting 


Type of programme — bachelor’s degree in accounting 

Level — undergraduate 

What’s of interest - implementation and evaluation of calibration 
meetings as a means to ensure comparable national standards 


This initiative to ensure comparable national standards in accountancy 
in Australia, engaged with 30 participants from 10 of 40 Australian 
universities representing a broad cross-section of business schools. 

Participants were senior accounting academics (two partici- 
pants from each university) and practitioners. Three iterations of the 
calibration activity were carried out in Darwin, Melbourne and Adelaide. 
They aimed to calibrate a threshold standard, namely, bachelor graduates 
will be able to: 


Justify and communicate accounting advice and ideas in 
straightforward collaborative contexts involving both accountants 
and non-accountants. (Watty et al. 2014) 


The participating universities provided anonymised samples of final year 
students’ written work. The calibration exercise involved three stages. 
Before the workshop, participants viewed samples online and rated the 
samples as to whether the threshold standard was met or not met; the 
project leaders felt having to decide between two grading categories 
would produce more agreement. Grades were submitted using an 
anonymised online system along with reasons for the grading decisions. 
After submitting grades and reasons, participants could then view other 
participants’ grades and comments, which were anonymised. Anonymity 
was important for participation. Viewing grades and comments helped 
assessors reflect on their own judgements and prepared them for the 
follow-up face-to-face workshops. 


MARKER RELIABILITY 


93 


The second stage was face-to-face workshops, attended by all 
participants. The workshops lasted three to six hours and consisted of 
dialogue around the samples and pre-workshop gradings. All reviews 
remained anonymous. Initially participants broke into small groups for a 
detailed discussion of the samples and threshold concept. This discussion 
helped develop a ‘shared understanding’ of the threshold standard and 
participants decided whether the sample met that standard. Finally, there 
was a plenary discussion with the aim of reaching a group consensus 
on which samples met the threshold and why. The third and final stage 
was a remarking of the samples, following the workshop discussion and 
within two weeks. 

The project team carried out an evaluation of the calibration 
process, collecting self-reported comments on the efficacy of the 
discussions. Participants felt that the process helped reduce variation 
and achieve more consistent assessment judgements: 


I think that the peer review process including the workshops 
themselves doe[s] help build a shared understanding of the 
standard recognizing that assessment of it is still a judgement 
call for all of us. Variation in assessments will definitely decline 
following the workshop in my view. (Watty et al. 2014, 472) 


Participants became more confident in their ability to judge national 
threshold standards after the second iteration: 


The Darwin workshop and, specifically, the social aspect of the 
moderation process appeared to foster this confidence. (Watty 
et al. 2014, 473) 


However, after the third iteration, participants gained an increased 
understanding of the complexities of assessment judgements and gaps in 
their own knowledge: 


However, and quite significantly, this confidence was not as 
prominent a theme as it was in the Melbourne pre-workshop data. 
There were also quite a few assessors who revealed scepticism 
about achieving consistency across multiple markers. (Watty et al. 
2014, 474) 


The aim of the calibration was not to achieve ‘perfect agreement’. The 
project team recognised that there are many reasons for variation in 
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assessor judgements (as discussed in this chapter). Their aim was to ‘to 
facilitate consistency and thus reliability and validity in marking’ (Watty 
et al. 2014, 472). 


Further Reading 
For further information about this case study see: 


Watty, Kim, Mark Freeman, Bryan Howieson, Phil Hancock, 
Brendan O’Connell, Paul de Lange and Anne Abraham. ‘Social 
Moderation, Assessment and Assuring Standards for Accounting 
Graduates’, Assessment and Evaluation in Higher Education 39, no. 
4 (2014): 461-78. 

See also a description of calibration activities in the Degree 
Standards project: Advance HE. “The Degree Standards Project’, 
(2019b). Accessed 10 November 2019. www.heacademy.ac.uk/ 
degree-standards. 


Implications for Practice: Ways of Developing Markers 


The considerable body of research into marking in higher education 
supports the claim that marking in the sector is unreliable. Lack 
of reliability stems from the role of tacit knowledge in assessor 
judgements, the impossibility of producing accurate written standards 
of complex student work, the ‘ill-structured’ (Voss and Post 1988) 
nature of assignments which aim to assess complex learning and the 
lack of opportunities for calibration of standards. Calibration as a way 
of improving reliability of marking judgements, across a programme 
and across a discipline has potential (Watty et al. 2014); calibration 
activities are being actively researched (O’Connell et al. 2016; Beutel 
et al. 2017). In the UK, calibration activities have been organised in a 
range of disciplines, e.g. geography and law, as part of the work of the 
Degree Standards Project (Advance HE 2019b). Workshops, calibration 
toolkits and case studies have been developed to provide guidance on 
how to do calibration (see Advance HE https://www.heacademy.ac.uk/ 
project-section/calibration-in-practice). 

Calibration can be used at programme team level, across a 
department, and across a discipline (at national level). Through sharing 
standards in a programme, team or across a department, marker 
judgements may be made more consistent and reliable, though variation 
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in judgement would still be expected given the complex assessment 
tasks that students undertake. In calibration activities, teams spend time 
discussing and agreeing standards before marking, so an ideal time to 
start this work is towards the end of the academic year. Team leaders 
can start by creating resources for sharing standards with teachers and 
students; for example, collecting and anonymising a range of student 
assignments that illustrate a standard or criterion. 

To organise calibration sessions, collect exemplars of students’ 
assignments, particularly around grade boundaries or work that 
illuminates a particular assessment criterion. Distribute to the 
teaching team and ask them to read and make a judgement about the 
assignments and give a rationale for that judgement (without using 
assessment criteria or rubrics); record these anonymously in a virtual 
learning environment. Either online or in a face-to-face meeting, discuss 
exemplars in groups and through discussion, try to tease out a list of char- 
acteristics used by the group to make a judgement about the standard 
of the exemplar. If agreement is reached, document this, recording the 
agreed characteristics. 

If agreement cannot be reached, try to identify the issues that 
prevented agreement, e.g. did the requirements of the assignment divide 
opinion? This can happen when students are required to make videos, 
write blogs and undertake other non-traditional forms of assessment. 
Assessors may disagree on the relative weighting to assign to the form 
(e.g. essay, video, web page) and the content of the assignment. 

Were there disciplinary differences? Did assessors have 
fundamental, epistemological disagreements? This can happen in inter- 
disciplinary programmes or in subjects that attract scholars from a variety 
of disciplines; for example, engineering may have teaching teams drawn 
from computer sciences, biomedical fields, mathematics and physics. In 
politics, teachers may come from history or the social sciences, and some 
may take a predominantly quantitative view of research, while others 
may adhere to qualitative paradigms with a constructivist underpinning, 
such as naturalistic inquiry (Lincoln and Guba 1985). 

Agreeing standards before marking, is more effective than relying 
on post-marking moderation (Bloxham 2009). However, calibration 
takes time and needs to be factored into workloads and planned within 
the programme schedule. There will always be variation in marking 
standards; there is no one correct grade for complex student work 
and variation in judgements is to be expected. However, as the studies 
cited above indicate, calibration has potential to establish more robust 
academic standards. 
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Follow-Up 


Reading 


For more on how markers use assessment criteria read: 


Bloxham, Sue, Peter Boyd and Susan Orr. ‘Mark My Words: The 
Role of Assessment Criteria in UK Higher Education Grading 
Practices’, Studies in Higher Education 36, no. 6 (2011): 655-70. 


For a detailed exploration of codifications of academic standards read: 


Sadler, D. Royce. ‘The Futility of Attempting to Codify Academic 
Achievement Standards’, Higher Education 67, no. 3 (2014): 
273-88. 


For a description and evaluation of calibration activity in Australia read: 


Watty, Kim, Mark Freeman, Bryan Howieson, Phil Hancock, 
Brendan O’Connell, Paul de Lange and Anne Abraham. ‘Social 
Moderation, Assessment and Assuring Standards for Accounting 
Graduates’, Assessment and Evaluation in Higher Education 39, no. 4 
(2014): 461-78. 


Investigating Your Practice 


Try calibration in your disciplinary team or link up with a disciplin- 
ary team from another institution to calibrate across teams. Begin by 
setting up a collaborative virtual learning environment, e.g. a Moodle 
or Blackboard, to collate marks and feedback on exemplar assignments 
and share with other teams. Follow the guidelines in this chapter for 
organising calibration and use the toolkits on the Degree Standards 
webpage (Advance HE 2019b). Evaluate the impact of calibration both 
on markers’ confidence levels and on variation of marking judgements. 
You should follow-up the calibration activity with an investigation of 
impact. Using interviews, a survey or reflective journals, you could 
investigate whether teachers feel more confident about their marking 
judgements. Or you could investigate whether the calibration activity 
brought about any changes in marking consistency in the department by 
examining first and second markers’ grades. Is there more agreement? 
Remember, however, that there may be many reasons for greater 
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agreement of grades (see the discussion in this chapter on factors that 
affect markers’ grades). 


Exploring Teacher Judgement 


Capture teachers’ judgements and what they value in student 
assignments. Begin by deciding what to calibrate, e.g. an assessment 
criterion or a borderline degree classification. Collect at least three 
exemplars of students’ work (ensuring that students give permission). 
Make sure you have a range. Anonymise the exemplars and ask colleagues 
to, without using assessment criteria or rubrics, explain what they value 
in the assignments, giving reasons for their assessment judgements; 
see the Stem Wishees website for examples of self-recordings from 
academics discussing what they value in students’ assignments 
(http://www.thinkingwriting.qmul.ac.uk/wishees ). 

Ask colleagues to record their reasons in writing and upload to a 
virtual learning environment. They may want to think about the below 
questions. 


e What they wanted students to do. 
e What they valued in each exemplar. 


Analyse the reasons given and what markers say they value. Are there 
common characteristics that all markers draw on? Do markers make 
similar judgements about the quality of students’ work but give very 
different reasons? What other issues can you identify? Share the teachers’ 
judgements with your colleagues and use to discuss the consistency of 
marker judgement in your team. 
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Chapter 7 
Peer and Collaborative Assessment 


This chapter explores the advantages and pitfalls of involving students 
in the assessment process. The importance of developing trust through 
developing students’ competency and integrity is highlighted (Carless 
2009) and there are examples of practical activities that aim to develop 
trust. 

In Chapter 6, I discussed academic standards, arguing that 
standards are constructed within the teachers’ community of practice 
(Sadler 2014; Butler Shay 2004). Teachers benefit from discussing and 
sharing their understandings of standards between each other, as well as 
sharing these understandings with students. Students need a good under- 
standing of academic standards to direct their learning. In this chapter, 
I describe how students can become fully involved in the assessment 
process, co-designing assessment tasks and making judgements about 
the quality of their own and their peers’ work. This involvement, and 
preparation for assessment design and marking, ensures that students 
have a good understanding of assessment processes and standards, are 
able to make judgements about their own and their peers’ work, can 
direct their own learning and plan strategies to develop their work. 


Why Involve Students in Assessment? 


When students are involved in the assessment process, they gain 
an understanding of the standards they are expected to achieve. In 
influential papers, Sadler (1989, 2009 and 2010) argues that students 
need to develop their understanding of assessment standards through 
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involvement in activities that replicate the experiences of markers (see 
also the discussion in Chapters 2 and 8 on evaluative judgement). 

Teachers develop their understanding of academic standards 
through their assessment experiences and interactions with peers 
(Bloxham and Price 2015; Sadler 2014; Butler Shay 2004). When 
teachers set an assessment task, they may not know initially what 
students are able to produce in response to the task, particularly a 
new assessment task. It is through seeing many student responses 
to assignments, seeing the range of work that students can produce, 
making judgements about that work and discussing these with peers 
that they develop a sense of standards on the programme (Sadler 
2010). In co-constructing academic standards, programme teams 
articulate what counts as good quality student work (see Chapter 6 
for a discussion on calibration). Sadler (2010) argues, that students 
need to experience a similar process; they need to see a range of work 
in order to develop an understanding of quality; by showing students 
a range of work, we expose them to the kind of experiences markers 
have. Through group discussion on the quality of this work, students 
are helped to develop an understanding of standards and an interpret- 
ation of assessment criteria. 


The overt aim is to shift the focus away from telling students about 
the quality of their work (disclosure) and towards having them 
see and understand the reasons for quality (visibility), and in the 
process develop personal capability in making complex judgements 
[...] educating students in the art of making substantive and 
comprehensive appraisals in ways similar to those characteristically 
used by expert assessors. (Sadler 2010, 546-7) 


If students have the opportunity to create constructions of academic 
standards and understand the standard they need to reach, they are less 
dependent on teachers telling (transmissive) them what to do (Sadler 
2010). This frees students from the need to check constantly with 
teachers or to be dependent on teachers for feedback on their work. 
Students can avoid ‘learned dependence’ (Yorke 2003, 489) on a teacher 
and become more autonomous learners. Through these experiences they 
develop assessment literacy which Price et al. (2012) define as a range of 
knowledge and skills, namely: 
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e an appreciation of assessment’s relationship to learning 

e aconceptual understanding of assessment 

e understanding of the nature, meaning and level of assessment 
criteria and standards 

e skills in self- and peer assessment 

e familiarity with technical approaches to assessment (e.g. 
familiarity with pertinent assessment and feedback skills [...] 

e possession of the intellectual ability to select and apply 
appropriate approaches and techniques to assessed tasks. 
(quoted from Price et al. 2012, 10) 


Teaching idea — Paraphrasing assessment criteria (Carless 2016) 


Assessment criteria is written in a professional discourse which is 
difficult for students to interpret (see Chapter 6). Check student 
understanding of assessment criteria by asking them to paraphrase 
the criteria in their own words. This can be done online or at the start 
of a seminar. Give students an exemplar of an assignment and the 
assessment criteria for the assignment. Ask students in pairs or small 
groups to take one criterion and rewrite the criterion in their own 
words. Share and comment on the rewritten criterion. This helps to 
demystify criteria and creates shared understandings of criteria. 

Collect and compare paraphrasing and discuss online or face-to- 
face. Does this activity reveal misunderstandings of the criteria? What 
are they? 

(See Carless 2016: https://tinyurl.com/tfr8k8j.) 

You could follow up this activity by looking at exemplars and 
identifying aspects of the criterion in students’ work, e.g. what does 
good quality analysis look like in students’ work? 

Follow this work with a guided marking activity. 


(See the explanation of guided marking in this chapter, below.) 


Students’ assessment literacy can be enhanced by involving them in 
assessment design and assessment judgements, but there are drawbacks. 
Students (and teachers) may not see this involvement as relevant to 
their studies. They may feel that they should be learning more content, 
more medicine or engineering. However, assessment is part of learning 
and needs to be fully integrated into the programme structure (see 
Chapters 3, 4 and 5). Activities to help develop students’ assessment 
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literacy, especially an understanding of academic standards, also need 
to be part of the programme; without an understanding of academic 
standards students cannot become self-directed learners (see box on 
previous page). 

Peer assessment is widely advocated in the literature but students’ 
perceptions of this vary. Students may view assessment design and 
marking as teachers’ work (Wilson et al. 2015) and, in the education 
market economy, they may feel short changed. In recent studies, 
students have expressed resentment about doing the teachers’ job and an 
increased student workload (Wilson et al. 2015; Van Zundert et al. 2010; 
Planas Lladó et al. 2014). They may also feel that they are not competent 
to take on these tasks and, if peer assessment is undertaken, may feel 
their peers are not competent. Preparing students and developing 
their competence is key to ensuring they understand and benefit from 
involvement in assessment. 


Establishing Trust 


Recently, many studies on peer assessment have raised concerns around 
issues of trust, fairness, friendship marking and peer pressure (Raes 
et al. 2015; Wilson et al. 2015; Cartney 2014; McConlogue 2012; Liu 
and Carless 2006). Students are concerned about their peers’ ability to 
give good quality feedback and the perceived laziness of peer assessors. 
Factors that militate against effective peer assessment are students’ 
lack of understanding of standards, the competitive higher education 
environment (Wilson et al. 2015), and modularisation of degree 
programmes, which creates difficulties in mapping assessment across a 
programme and building assessment literacy. 

Wilson et al. (2015, 19) have called for a more ‘nuanced’ view 
of peer assessment, recognising the ‘realities’ of peer assessment in 
practice. Student resistance to peer assessment is particularly strong 
when peer assessment involves giving summative grades, and less 
resistant when students are involved in formative peer review activities. 
There is wide agreement that students need to be prepared for peer 
assessment (Topping 2010; Orsmond 2004; Falchikov 2004). Sluijsmans 
et al. (2002) have argued for an extended preparation period — not just 
a one-off workshop. Some of the common issues in peer assessment can 
be found in Table 7.1. 
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Table 7.1: Peer assessment issues (sources: Liu and Carless 2006; 
Cartney 2014; McConlogue 2012; Wilson et al. 2015) 


Peer assessment issues 


Peers give widely different grades (summative assessment) 


Peers give inconsistent feedback — poor quality feedback 


Laziness of peer assessors 


Peers’ lack of understanding of standards 


Competitive HE environment 


Friendship marking 


It is clear from these studies that good preparation of students is 
essential; before involving students in assessment, the teacher needs to 
ensure they have a good understanding of academic standards on the 
programme and that they have had previous practice of designing tasks, 
working collaboratively, making judgements about peers’ work, writing 
and responding to feedback and self-assessing. 


Developing Student Competency and Integrity 


Competence and integrity are seen as key to successful peer assessment. 
Carless (2009) argues that students need to acquire competency and 
integrity in order to perform the role of peer assessor. Competent peer 
assessors understand the required academic standards and are able 
to make judgements about the quality of peers’ work, with integrity. 
They can also give good quality feedback and respond constructively to 
peers’ feedback comments. For students to be competent, they need to 
understand academic standards and have enough experience assessing 
work in the relevant setting so that they can make a judgement about 
the standard of that work and self- and peer assess. In successful peer 
assessment, students demonstrate integrity and put effort into giving 
good quality feedback; when students feel their peers have not put effort 
into giving feedback, they lose confidence in their peers’ judgements. 
Lack of integrity leads to ‘anger’ towards students who do not fully 
participate in peer assessment (Cartney 2014, 559). Competent peer 
assessors have acquired a good understanding of the purposes and 
processes of assessment and assessment standards, are able to make 
judgements about the quality of work and compose and use feedback to 
move on in their learning. 
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Drawing on the research literature, I propose four key principles 
to consider when developing students’ understandings of academic 
standards. 


1. Students need to see a range of their peers’ work. This may be 
exemplars from a previous year or a similar assessment. Sadler 
(2010) argues that student experiences need to mirror teacher 
experiences in order to develop a good understanding of quality; 
teachers see a wide range of student work and this helps them 
develop an understanding of quality. 

2. Students need to articulate what they value in these exemplars, e.g. 
through oral or written comments. Articulating what they value is 
a powerful learning experience. Composing feedback is cognitively 
demanding and helps students develop assessment judgements as 
they assess peers’ work (Nicol 2010; Nicol et al. 2014; McConlogue 
2015). 

3. Students discuss their value judgements with others and engage 
in dialogic feedback (Carless and Boud 2018) with peers and 
teachers. Through discussion in a community of practice, students 
compare their understandings and negotiate socially constructed 
standards within the group. (See Sadler 2014, Butler Shay 2004 
and Chapter 6 for more on the socially constructed nature of 
academic standards.) 

4. Students develop confidence in their peers’ ability to assess and give 
feedback; peers are perceived as having good levels of assessment 
literacy (competency) and are trusted to give detailed and helpful 
feedback (integrity) (Carless 2009). 


These principles can be used to design assessment activities to develop 
students’ understanding of assessment. The flexible framework 
proposed in this chapter suggests ways of gradually introducing students 
to peer assessment, building expertise across a programme of study. 
A key aspect of the framework is ensuring that students are gradually 
introduced to the notion of academic standards and acquire experience 
in making assessment judgements. In the framework it is assumed that 
when students feel that their peers are competent, they are more likely to 
value peer feedback. If levels of trust in the group are not high, students 
can still develop their understanding of academic standards without 
engaging in peer assessment through engaging in stage one preparation 
activities (see below). 
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Activities for Developing Competence and Integrity 


Stage One: Preparation Activities 


Falchikov (2004) points out the importance of preparation in getting 
students onboard. Preparation activities occur at the beginning of the 
programme, when students are in most need of checking their under- 
standing of assessment standards and are most likely to accept working 
with peers as the ‘norm’ for this programme of study (Rudy et al. 2001). 
The aim is to move students away from relying on authoritative teacher 
feedback at an early stage, thus helping students to avoid ‘learned 
dependence’ (Yorke 2003, 489) on teacher feedback. 


Guided Marking 


This is a widely-used strategy, designed to help students explore and 
gain an understanding of teacher expectations and standards. Students 
read a range of written work from a previous year or view recorded 
presentations. The previous year assignments show a range of quality 
(for example, assignments judged by the teacher to be satisfactory, good 
and excellent) so that students see a range of possible responses to the 
assignment brief and can begin to make judgements about the quality 
of work (principle one above). Guided marking is organised either in a 
seminar or lecture, or online, before students start their own assignments 
so that their experience of reading and discussing these assignments 
helps them to understand teacher expectations. Students articulate their 
judgements in feedback comments (principle two above). It is crucial 
that the teacher organises online or face-to-face discussions about the 
judgements students make, as this gives students the opportunity to 
explain their judgements, raise issues and question the teacher about 
his/her judgement and what s/he values in the assignment (principle 
three above) — see box on following page. 


Marker’s Commentary 
Explaining standards is difficult; showing a range of exemplars and 
explaining why the exemplars meet the standard helps students better 


grasp what they need to aim for. In a marker’s commentary, teachers 
comment on past assignments explaining their assessment judgements 
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Teaching idea — How to implement guided marking 


Organise guided marking at the start of a module, as it helps students 
understand the standard of assignment required. 

Start by collecting exemplar assignments from previous year 
students and gain permission to use these exemplars. If the module 
is new, use similar assignments from a related module. Explain the 
assignment to students either online or in class. Students then receive 
assessment guidelines and three exemplars that show a range of work 
(borderline to excellent); it is important that students see a range as 
this helps develop their understanding of quality (Sadler 2010). 

Give students sufficient time to read, rank, give grades and 
make comments on the exemplars. This can be done online in a 
virtual learning environment, e.g. using a Moodle workshop (see 
below and https://docs.moodle.org/22/en/Workshop_module for a 
description of how to use Moodle workshop). Either online or in class, 
lead a discussion on the exemplars, comparing grades and comments. 
Prompt students to discuss key issues, e.g. presentation of results, 
criticality and creativity. It helps students to interpret standards and 
assessment criteria if you can point to examples of these criteria in the 
exemplars, explaining what you value in student work. Finally, give 
students practice in writing constructive feedback on the exemplars; 
they can post and share feedback online and you can also share your 
feedback comments. Lead a discussion (online or face-to-face) on 
what feedback students find helpful and why. 

This activity helps students, especially first year students, to 
better understand what good quality work looks like in their discipline. 
It also helps you to explore and guide their thinking and share your 
knowledge of academic standards. Be prepared for lively discussion 
and for challenges to your grading and feedback comments. 


of exemplar assignments, discussing what they value in the work and 
explaining difficult concepts like ‘criticality by pointing out concrete 
examples in the work: ‘this is what I mean by criticality and here’s an 
example in this assignment’. See the Stem Wishees website for examples 
of marker’s commentaries (http://www.learningdevelopment.qmul. 
ac.uk/wishees). In recorded videos, teachers use exemplars to clarify 
their expectations and explain assessment standards. 
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A follow-up discussion is organised, e.g. through a seminar or 
online forum, so that students can ask questions and critique the 
teacher’s judgements (dialogic feedback, principle three above). 


Teaching idea — How to make a marker’s commentary 


Use a marker’s commentary to reinforce the guided marking activity or 
as a replacement. Marker’s commentaries can be done wholly online 
or blended; students can read exemplars and listen to a recorded 
commentary online and discuss later in a lecture or seminar. To make 
a marker’s commentary, first collect examples of a range of student 
assignments for your module, with permission to use, and anonymise. 

Think about the assignment task you set and what you expected 
students to achieve. It is helpful to think about what you value in 
students’ work; this may be criticality, creativity or a demonstration 
of professional competencies. Look for examples of these qualities 
in the range of exemplars. Make a short video or audio recording 
or write a short commentary on what you value in the exemplars, 
clarifying your expectations. You may want to listen to examples 
of teachers doing this on the Stem Wishees website (see http:// 
www.thinkingwriting.qmul.ac.uk/wishees/collections/quinnipiac/ 
microbiologyundergraduatereport/55529.html for an example). 
Post the written or recorded (audio or visual) commentaries online. 
Encourage dialogue; ask students to discuss the exemplars and 
commentaries. This can be done anonymously using an online 
forum. Follow up the discussion and post frequently asked questions 
(FAQs) with responses. Monitor use of the marker’s commentary 
and use student feedback to improve your commentaries and clarify 
misunderstandings. 


Peer Review 


In peer review, students are provided with opportunities to articulate 
their judgements and try out their new understandings of teacher 
expectations by reading their peers’ written assignments or listening to/ 
watching oral presentations and giving feedback. Principle two states 
that students articulate their judgements, either orally or in a written 
form. There is evidence that composing feedback is beneficial, and 
perhaps more beneficial than receiving it (Nicol et al. 2014; McConlogue 
2015; Kim 2009). 


PEER AND COLLABORATIVE ASSESSMENT 


107 


108 


As giving feedback to peers is one area of contention (because of 
power relations and friendship marking (Liu and Carless 2006; Cartney 
2014)), in order to maintain good group relationships this feedback is 
initially written for the teacher, and not shared with peers. In composing 
feedback, students articulate their understanding of assessment 
standards, giving the teacher some insight into what students understand 
and what they struggle to understand. Teachers comment and respond 
to student feedback either individually or, depending on class size, with 
generic feedback, explaining what they thought students did well, ways 
they could develop their feedback and what they should comment on. As 
students do not receive their peers’ feedback, the conflict in the group 
that can arise is avoided while students develop expertise in articulat- 
ing feedback. Students can use insights from peer review discussions to 
self-assess their work. 


Using Preparation Activities 


Depending on the students’ prior expertise, preparation activities could 
be performed several times at the beginning of a programme of study 
(e.g. throughout the first year of an undergraduate programme, or the 
first term of a postgraduate programme) so that students gradually build 
up expertise. There is evidence that, over time, the quality of students’ 
feedback improves as their understanding of disciplinary constructs 
develops (Wen and Tsai 2008). As an added bonus, students can use 
their growing expertise to self-assess their own work before submitting, 
thus directing their own development. 

After each of these activities, the group debriefs, evaluating how 
well the activity worked and what the group gained from it. Lessons are 
learned for the next iteration. The framework is flexible and designed to 
be adapted to any educational context; the teacher and students decide 
whether they want to progress to the next stage. They may decide to 
stay with stage one activities as these cover the first three principles for 
developing understandings of academic standards. 


Stage Two: Practising Activities 


Once students have sufficient experience of preparation activities, the 
group takes a decision as to whether to move on to practising activities. 
Depending on the context, some groups may feel most comfortable 
and derive the most benefit from stage one activities, and may wish to 
continue with these while other groups may move on quickly to stage 
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two and three activities. (See below, How the framework might be used, 
for more suggestions.) 


Peer Review and Exchange of Comments 


Peer review of summative activities is obviously higher risk than peer 
review of formative activities and necessitates a level of competence, so 
that assessment feedback is perceived as reliable (or at least as reliable 
as teacher feedback). A level of integrity is developed so that competition 
and friendship marking (Wilson et al. 2015; Carless 2016) do not affect 
the feedback. Multiple peer assessors can help develop judgement as 
students see multiple views of their work and this can support students 
in composing a response (a rebuttal) to peer reviewer comments. 

At this stage of the framework, students initially start a peer review 
of draft assignments or practice oral presentations. Students receive 
feedback on draft work or on practice presentations and then use this 
feedback to improve the final assignment before submitting for teacher 
marking. By this stage, the first three principles have been met, and the 
exchange of feedback comments has begun the process of developing 
trustful relationships within the group (principle four). 


Peer Assessment (Summative) 


For summative peer assessment, students need to have confidence in 
their peers’ competence and integrity. In summative peer assessment, 
students award grades as well as providing feedback. Grading work, 
rather than just commenting, is more contentious and causes more 
student protest. Commonly reported problems are lack of engagement, 
poor quality feedback (leading to lack of trust in the peer assessor) 
and friendship marking (McConlogue 2015; Cartney 2014; Wilson et 
al. 2015). To minimise contention, peer assessment is introduced on 
small tasks, e.g. a background section of report (McConlogue 2012), 
which carry a small percentage of the overall final summative grade. 
After submission of the assignment, students attend a rehearsal marking 
(Falchikov 2004), where they are provided with a sample range of the 
assignments to be assessed; rehearsal marking can also be set up online. 
In the rehearsal marking, students have an opportunity to ask questions 
about how any difficult concepts or calculations should be marked. 
Following principle one, they see and grade a range of assignments. 
They discuss their grading in groups and with the teacher, and practise 
composing feedback comments. Students are now prepared to peer 
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assess assignments. Assignments are allocated to students and, within 
an agreed deadline, they grade and write feedback comments which are 
automatically returned. Students receive their grades and feedback and 
then write a rebuttal to the peer reviewers. 

As students develop the ability to give good quality feedback, the 
group decides whether to peer assess larger and more heavily weighted 
assignments. A requirement for moving on to the next stage is that 
teachers and students are confident of their own and their peers’ ability to 
give reliable grades and feedback. To further establish trust, moderation 
and complaints procedures are set up. Friendship marking and grader 
reliability can be monitored, especially if an online system is used (e.g. a 
Moodle workshop, see Wilson et al. 2015 for an explanation). Complaints 
procedures provide a safety valve for dissatisfied students. Commentary 
on peer feedback, or a rebuttal, where peers explain whether the 
feedback was helpful and what they will use from it, mirrors the peer 
review process. The rebuttal provides a safety valve for students who feel 
disgruntled with their reviewers’ comments and ensures that students 
take review seriously and give responsible comments. The rebuttal also 
helps reviewers reflect on the comments they compose and how they 
might be improved, thus developing their expertise. After each of these 
stages, the group debriefs, evaluating how well the activity worked and 
lessons learned are used to develop the next iteration or progression to 
the next stage. 


How to Organise and Moderate Peer Assessment 


Good organisation and preparation are essential for successful peer 
assessment (see Figure 7.1). Follow the guidelines for preparation above; 
for example, guided marking and rehearsal marking. Once students are 
ready to peer assess, ensure that their assignments are submitted and 
anonymised, e.g. use an online tool like a Moodle workshop (see below 
and https://docs.moodle.org/22/en/Workshop_module). Allocate peer 
assessors, ensuring each student receives multiple assessors; a Moodle 
workshop can be set up to automatically allocate multiple assessors. If 
possible, group assignments according to level and ensure peer assessors 
receive a range. Provide a feedback worksheet for guidance and/or 
feedback comments (ensure both have been used and discussed in 
preparation activities — do not introduce new worksheets at this stage). 
Ensure that students are aware of the moderation and complaints 
procedures. You need to be aware of any issues that arise during peer 
assessment so that you can resolve them quickly. Give peer assessors 
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a deadline (e.g. two weeks). A Moodle workshop can be configured 
to automatically end access by the deadline and distribute grades and 
comments. Ensure you check grades and comments before they are 
made available to students; some software can help by identifying the 
spread of peer assessors’ grades. Resolve any complaints and evaluate 
peer assessment using your reflections and student feedback to improve 
the next iteration. 


Preparation Explanation of moderation and complaints | 
e.g. Guided marking and rehearsal marking procedure 


Deadline e.g. 2 weeks for return of grades Anonymised assignments allocated to peer 


and feedback. Strictly enforced. assessors 
Teacher moderation of grades and Distribution of marks and grades to 
feedback (identify any issues) students with evaluation questionnaire 


Group debrief. Analyse evaluation Complaints procedure activated if 
responses and identify lessons learned for necessary 
next iteration 


Figure 7.1: Organising peer assessment 
Using Software to Manage Peer Assessment 


Software can help enormously with the administration of peer 
assessment, especially in large classes. For example, Moodle can be 
configured to (adapted from Moodle 2019): 


- set up practice ‘rehearsal marking’ activities; students mark 
exemplar assignments and compare with teacher’s judgements 

- collect submitted assignments 

- anonymise and allocate assignments to peer markers 

- host ‘structured assessment forms’, feedback worksheets, 
assessment criteria and ‘rubrics’ 

- send reminders to peer assessors and close down when the 
deadline is reached 

- collect and average marks from multiple markers 

- permit teacher moderation of grades; teachers can ‘overide’ 
average peer assessor grade, if judged to be necessary 

- distribute grades and feedback to students. 
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(Adapted from Moodle 2019 Workshop Module: https://docs.moodle. 
org/22/en/Workshop_module. Also see https://docs.moodle.org/35/ 
en/Using_Workshop#Workshop_ grading for an explanation of grading.) 


Stage Three: Partnership 


Once students have completed preparation and practising activities 
(and how long this takes will vary depending on students’ prior 
knowledge and experience) the group is ready to move on to the next 
stage, to collaborative assessment. The judgement about where to start 
and what to move on to is increasingly taken within the group with 
students and teacher collaborating, building a trustful relationship. 
For example, in a professional programme, the programme may start 
with stage one activities in the framework and move quickly through 
to stage three. 

At the partnership stage, students have considerable expertise in 
making assessment judgements and composing, evaluating and acting 
on feedback. They also gain control over designing the assessment 
task, co-constructing assessment criteria, giving feedback and grades 
(McConnell 2002, 2006). This stage requires a high level of competency 
and integrity; the group needs to agree academic standards. The 
teacher’s role is to facilitate group discussions, provide guidance on time 
commitments, mediate, and ensure student-developed assessments meet 
quality assurance and institutional requirements. To be able to handle 
this stage, students need to be acculturated into academic standards and 
assessment processes in the institution, and have developed expertise 
in making assessment judgements. As above, moderation and student 
complaint procedures should be clear. 


How the Framework Might be Used 


In an undergraduate degree, this framework could be used to develop 
understandings of academic standards across the programme, starting 
with stage one activities in the first year and progressing, if the group is 
ready, to stage three by the final year. Taught postgraduate programmes 
and continuing professional development courses, might begin with 
guided marking (stage one), but progress quickly through the stages 
and introduce collaborative assessment early in the programme or in 
the second semester (see McConnell 2006 for an example of this). As 
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groups develop differently, interact differently, and as building a trustful 
relationship is especially essential to stage three of the framework, it 
is important the group makes a judgement about how fast to progress 
through the stages. 

The institutional structures for quality assurance may present 
problems; for most programmes, the assessment diet is set and approved 
at the outset of the course. In order to involve students in decisions about 
the framework stage(s) most appropriate for their cohort, there would 
need to be flexibility so that these decisions could be made and reviewed 
throughout the programme, depending on the cohort’s needs, how they 
get along together and how quickly their expertise in giving constructive 
feedback develops. 

Activities that develop students’ understanding of academic 
standards and provide lots of opportunity for dialogue, so they can 
compare their constructions with peers’ and teachers’ constructions, 
need to be included in the programme design. Figure 7.2 summarises the 
stages and relevant activities and corresponding assumed development 
of competency and integrity. 


Collaborative 
assessment. 
Peer Design task, 
assessment assessment 
Competency (grading criteria and 
and integrity Peer review 4 grade, and 
with i 
Peer review, comments). 8!VE 
, exchange of feedback 
noexchange comments Rebuttal 
: of comments. 
Guided Teacher 
marking. feedback on 
Marker’s comments 
commentary 
Stage 1 Stage 2 Stage 3 
Activities 


Figure 7.2: Stage 1-3 activities (preparation to partnership) 


The following case study discusses collaborative assessment in an online 
master’s programme, illustrates student involvement in co-designing 
tasks, constructing assessment criteria, giving feedback and agreeing 
grades. 
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Case Study: Peer and Collaborative Assessment in an Online 
Master's Course 


Type of programme — MEd in e-Learning 

Mode - fully online 

Level — postgraduate 

Cohort — home and international students 

Learning approach - collaborative learning community (community of 
practice, democratic approach to learning and power relationships) 
What’s of interest — use of collaborative assessment; students design the 
assessment tasks, specify standards collaboratively, devise assessment 
criteria, work in groups or individually and carry out self-peer-teacher 
review of assignments. 


This fully online master’s programme recruits students interested in 
e-learning, typically education professionals, e.g. teachers and librarians. 
The programme is run entirely online, with students working collabora- 
tively in groups, or with a teacher, in small learning sets. The groups 
communicate through conferencing software with areas for social chat 
and areas for collaborative and co-operative learning. At the start of the 
programme there is an initial ‘socialising’ stage, where students exchange 
information and start to form a community (Brauer et al. 2019; Salmon 
2010, 2019). For example, they may write a professional story, reflecting 
on the key influences on their practice and share appropriate sections 
with the group. Throughout the programme, the emphasis is on students 
working together, either collaboratively on student-designed tasks, or 
co-operatively on individual tasks, sharing discussions and supporting 
each other. Assessment is seen as an integral part of learning on the 
programme; assessment is ‘part of the content of the course’ (McConnell 
2006, 93). The learning community communicates through online fora, 
divides into smaller learning sets to work on specific projects and tasks, 
and comes together in online workshops. The action research approach 
to learning in the programme means students are involved in a range of 
small-scale research projects and in discussions and critique of research 
paradigms and methodologies. 


The Role of the Student 


Students work as part of a learning community and are required to 
participate actively in online discussions, to share, discuss and support 
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each other’s learning. Working together, the groups devise and negotiate 
assessed tasks. In designing the tasks, students also discuss what 
academic standard their work needs to reach and agree the design 
of the task and assessment criteria. This collaborative construction 
of the tasks and mutually agreed assessment criteria helps students 
better understand academic standards on the programme. Groups are 
able to choose the assessment topic (see Chapter 9 for a discussion of 
assessment choice and inclusive practice) and receive support from their 
peers and the teacher. In asynchronous online discussions, they explore 
ideas and share resources. Drafts are self-peer-teacher reviewed; the 
student first reviews their own work, identifying areas for development 
and requesting support. Peers and the teacher work to help the student 
with suggestions for ways of developing and enhancing their work. 


The Role of the Teacher 


The teacher guides students and mediates between student desires and 
institutional regulations. S/he ensures that the assessment tasks are valid 
and of the appropriate level and feasible within the module timeframe. 
The democratic approach to power relationships in the programme 
means that the teacher’s review and assessment of student work can 
be challenged by individual students or the group as a whole. The final 
grade for any assignment is agreed across the whole learning community 
(which includes the teacher). 

Students are, on the whole, very positive about experiencing 
collaborative assessment. One student commented: 


The assessment process is a lot more integrated into the whole 
learning process. Instead of being something ‘out there’ and 
threatening, it can actually be a supportive and motivating process. 
(McConnell 2006, 140) 


Others commented on the difficulty of offering criticism that might 
offend. One commented: 


I wouldn’t exactly say ‘too kind’ but each peer reviewer has come in 


with very, very constructive remarks indeed ... (McConnell 2006, 
132) 
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While another expected the teacher to intervene: 


I think saying the really hard things is the responsibility of the 
teacher, because despite the fact that I know how keen you (the 
teachers) were in dismantling those differences, the bottom is not 
possible, and there were times when I thought ‘no I want teacher 
intervention here’ ... (McConnell 2006, 127) 


Further Reading 


For a full account of this programme and evaluation data, see the titles 
below. 


McConnell, David. E-Learning Groups and Communities. 
Maidenhead: Open University Press, 2006. 

McConnell, David. ‘The Experience of Collaborative 
Assessment in e-Learning’, Studies in Continuing Education 24, no. 
1 (2002): 73-92. 


For research on online socialisation see: 


Brauer, Sanna, Anne-Maria Korhonen and Pirkko Siklander. ‘Online 
Scaffolding in Digital Open Badge-Driven Learning’, Educational 
Research 61, no. 1 (2019): 53-69. 

Salmon, Gilly. ‘The Five Stage Model’. Accessed 10 November 
2019. www.gillysalmon.com/five-stage-model.html. 

Salmon, Gilly, Ming Nie and Palitha Edirisingha. ‘Developing 
a Five-Stage Model of Learning in Second Life’, Educational 
Research 52, no. 2 (2010): 169-82. 


Follow-Up 


Reading 


To learn more about the pitfalls of peer assessment, read: 


Wilson, Michael John, Ming Ming Diao and Leon Huang. “Tm Not 
Here to Learn How to Mark Someone Else’s Stuff”: An Investigation 
of an Online Peer-to-Peer Review Workshop Tool’, Assessment and 
Evaluation in Higher Education 40, no. 1 (2015): 15-32. 
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For more on trust and integrity, read: 


Carless, David. ‘Trust, Distrust and Their Impact on Assessment 
Reform’, Assessment and Evaluation in Higher Education 34, no. 1 
(2009): 79-89. 


For an investigation and evaluation of collaborative learning, read: 


McConnell, David. E-Learning Groups and Communities. 
Maidenhead: Open University Press, 2006. 


Investigating Your Practice 


Introduce guided marking in your programme, especially at the 
beginning of core modules. Collect students’ perceptions of what they 
feel they learned from the guided marking activities and monitor 
performance. Discuss with colleagues. What impact has guided marking 
had (if any) on students’ performance and understanding of standards. 
Can you see any evidence of development in their work, e.g. are research 
results more clearly presented? Are literature reviews more critical? 

Work with students and colleagues to redesign assessment on 
your programme and introduce activities to prepare students for peer 
assessment. Plan stage one activities (see above) at the beginning of your 
programme, across several core modules. You could create a through- 
line of assessment activity across the programme (see Chapter 4), with 
staged introduction of preparation and practising activities (stages 
one and two above). Evaluate the effectiveness of these activities with 
students, e.g. student facilitated focus groups, and staff (through, for 
example, conversations, see Kvale 1996). Identify lessons learned and 
refine activities for subsequent years. 
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Chapter 8 
Giving Good Quality Feedback 


Students express dissatisfaction with feedback in higher education in 
student satisfaction studies, e.g. the National Student Survey (Office for 
Students 2019) and Postgraduate Taught Experience Survey (Advance 
HE 2019a). This chapter investigates why students are dissatisfied with 
teacher feedback and suggests ways of improving feedback. I consider 
how to ensure that feedback is of good quality, and that students 
understand and can make use of feedback. I discuss the importance of 
involving students in the feedback process, developing student ability 
to judge standards of their own and peers’ work and developing the 
student’s ability to self-assess and become an independent learner. 
Students need opportunities to learn from dialogue around feedback, 
with peers and teachers. Composing feedback on peers’ work enhances 
students’ understandings of feedback, enabling them to self-assess their 
own work and direct their learning. The chapter will also look at the 
problems of involving students in assessment and will emphasise the 
importance of preparation addressing the question of how to develop 
evaluative judgement (Tai et al. 2018). 

Boud and Falchikov (2007) argue that ‘assessment rather than 
teaching, has a major influence on students’ learning. It directs attention 
to what is important. It acts as an incentive for study. And it has a 
powerful effect on what students do and how they do it’ (2007, 3). Some 
assessment practices are focused on students demonstrating what they 
know, rather than focusing on the learning process and how students will 
continue to learn after they leave the institution. Sustainable assessment 
involves students completing tasks which equip ‘students to learn for the 
long term’ (Boud and Falchikov 2007, 5), developing knowledge and 
skills that can be used after their higher education studies, so that when 


they are faced with life and professional challenges, they can draw on a 
‘repertoire of assessment-related practices’ (Boud and Falchikov 2007, 5) 
that they have developed to help them problem solve. 

Assessment is highly regulated and is often equated with quality 
assurance and measuring outcomes rather than promoting learning. 

Students are constructed as ‘passive subjects’ (Boud and Falchikov 
2007, 17); assessment is something that is done to them. Sustainable 
assessment requires students to be active in assessment, determining 
‘what is to be learned, how it is to be learned, and how to judge whether 
they have learned it or not’ (Boud and Falchikov 2007, 18). Similarly, 
students are often passive recipients of feedback messages from teachers 
and may not understand or may misunderstand those messages. This 
creates ‘learned dependence’ (Yorke 2003, 489) on the teacher. To 
move away from learned dependence, students need to learn how to 
make judgements and have practice in evaluating evidence, appraising 
situations and drawing conclusions (Boud and Falchikov 2007, 19). 


Why are Students Dissatisfied with Feedback? 


Dissatisfaction with feedback has been reported in the literature and in 
student satisfaction surveys, e.g. NSS (Office for Students 2019) and 
PTES (Advance HE 2019). Dissatisfaction centres around the timing of 
feedback, feedback quality and student ability to use feedback. Some key 
feedback issues include the following. 


° Slow turnaround times and dissatisfaction with the quantity of 
feedback given and/or generic feedback (feedback given to the 
group, not to the individual). 


The massification of higher education in some countries has led to 
large class sizes (e.g. ~700 students). Dialogic feedback is individual, 
personalised, open to discussion and clarification, and supports students 
in planning their learning development. Large classes need increased 
resources in order to give dialogic, individualised feedback to students. 
With large classes, feedback turnaround times can be more than four 
weeks, and the longer the turnaround, the more feedback becomes 
increasingly less relevant to students who have moved on to tackling 
other assessments. Alternatives such as generic feedback and the use 
of peer assessment, which can decrease turnaround times, can often 
produce hostile reactions from students (Wilson et al. 2015), who feel 
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that their individual needs are not being met. See Chapter 7 for a detailed 
discussion of peer assessment. 


e Feedback is not useful for the next assignment. 


This is both a feedback issue and a design issue. Students need to hear or 
generate suggestions on ways of developing their work, and they need 
to be able to act on these suggestions in their assignment. Assessments 
often occur at the end of a module, so feedback needs to be relevant to 
the assignment in the next module. If there is no opportunity to use the 
feedback immediately, it becomes irrelevant (Price et al. 2010). Teachers 
report students not picking up feedback and the opening rates for 
feedback in virtual learning environments can be low. One inference may 
be that students do not find feedback useful for their next assignment so 
do not bother to collect or open it. 


° Students do not understand the feedback given. 


It is essential to extend higher education to groups that have tradition- 
ally been excluded and have not had the opportunity to study in higher 
education. Non-traditional students in higher education require changes 
in traditional curricula to address their needs. Support is beneficial for 
all students, especially students in diverse cohorts with diverse needs, to 
help them understand ways of studying and thinking within disciplines, 
and facilitate understanding of academic standards. If students do not 
understand standards and teacher expectations, they are unlikely to be 
able to make sense of teachers’ feedback or understand how to use it to 
develop. In essence, teachers spend a lot of time composing feedback 
which students may not understand or may not be able to use (Ivanič 
et al. 2000; Price et al. 2010). It seems student bewilderment about how 
to use feedback to improve their work fuels dissatisfaction. 


° Students feel over-burdened with assessment (Harland et al. 2015). 


Increasing use of modularisation, standalone modules and smaller 
modules has exacerbated the problem of overassessment in higher 
education. A common practice is to have two pieces of assessment for 
each module; only one assessment increases the risk of students doing 
badly on the module or failing. Two or more assessments provide students 
with more opportunity to demonstrate knowledge and skills, but also has 
the consequence of substantially increasing the number of assessments, 
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especially if the number of modules has been increased. The assessment 
load puts pressure on teachers and students; it becomes difficult to find 
time to cover all the module content and find time to prepare students for 
different types of assessments. Students faced with an increased amount 
of assessment have no time to do formative assessments, or to have 
dialogue around feedback with peers and teachers. Increased assessment 
can force students to become strategic learners, seeking cues on ways 
to cut corners and achieve good marks. The burden of assessment on 
students and teachers seriously impacts on good quality learning (see 
Chapter 4 for more on overassessment). 


e Dependency on exams. 


In response to large class sizes and concerns around plagiarism of 
coursework, essay mills and contract cheating (Medway et al. 2018; 
Amigud and Dawson 2019) exams have become more widely used. 
Contract cheating entails students submitting work that they have paid 
someone to write for them; this can entail either buying an essay from 
a website or paying a contractor to write an assignment. Similarity 
software cannot detect contract cheating if the contractor has written 
original content. Concerns over the growth of websites offering such 
services has led some institutions to prefer a greater weighting of exams 
over coursework assessment. While some students prefer exams, others 
may be disadvantaged by an overreliance on exams and there is no 
guarantee that exams will reduce cheating. With increasing concern over 
student mental well-being in higher education, increased stress needs to 
be avoided. Exams often occur at the end of a module and feedback, if it 
is given, may not focus on how students can develop their learning. 


° The affective dimension. 


Feedback can have an adverse effect on student learning if the student 
perceives the feedback to be critical, negative and a comment on their 
knowledge. The emotional aspect of feedback is under-researched, 
but studies suggest that power dynamics in higher education can be 
destructive; students perceive teachers as authoritative and negative 
criticism from teachers can demotivate students. Ryan and Henderson 
(2018) suggest that international students and students who receive 
feedback that does not meet their expectations are most at risk of negative 
responses. In a study of over 4,500 students, they found that 42.6 per 
cent of international students felt discouraged by feedback, compared 
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to only 35.4 per cent of domestic students. Students who receive lower 
than expected grades also experienced negative emotions. The authors 
conclude that feedback dialogues and developing trust between students 
and assessors can support students to learn from feedback. 


° Use of (unsupported) teaching assistants (TAs). 


As teachers become more pressed for time and accumulate larger marking 
workloads, they may look for help from teaching assistants, typically 
postgraduate students. TAs can give excellent feedback as they are closer 
to the student experience but they need to be supported, mentored 
and offered professional development; in addition their work needs to 
be moderated. It is important to ensure TAs have clear understandings 
of academic standards, that they understand the role of feedback, are 
aware of the effects of destructive feedback and have time to engage in 
dialogic feedback (Ryan and Henderson 2018). TAs can make a valuable 
contribution to a programme, but only if fully supported; this support 
and mentoring of TAs creates extra workload for teaching staff so more 
resources need to be available. 


e Quality and consistency of feedback. 


Academic departments, especially in research intensive institutions, 
may not make time to discuss standards and compare feedback given by 
markers. Without discussion, teachers can develop their own (possibly) 
idiosyncratic views of assessment standards and feedback. This can 
happen with experienced and inexperienced teachers. Consistency of 
feedback messages can be better achieved through dialogue between 
teachers and between teachers and students, checking understanding 
and socially constructing standards (Sadler 2014) through calibration 
activities. See below for a discussion of peer review of feedback and see 
Chapter 6 for an explanation of socially constructed academic standards. 


° Too many/too few varieties of assessment. 


Diversity in assessment needs to be carefully managed. Too few varieties 
can disadvantage some students. See Chapter 2 for a discussion of essayist 
literacy and how the essay can disadvantage non-traditional students 
(Lillis 2001). However, too many assessment varieties — blogs, vlogs, 
reflective pieces, quantitative lab reports, posters and oral presentations 
—may confuse students who need to have practice in different varieties of 
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assessment (Gibbs and Simpson 2005). Good practice involves designing 
a range of carefully planned assessments, linked over a programme. 
Often, assessments are not connected across a programme; there is no 
through-line of assessment activity so that students can take learning 
from one assessment into the next. Assessments need to be planned so 
that students get practice in assessment varieties and can learn how to 
get good at each variety before they are summatively assessed. Practice 
in assessment is vital for diverse student cohorts, as is the opportunity to 
choose assessment tasks so that students can demonstrate learning in a 
way that plays to their strengths (see Chapter 9 on inclusive assessment). 


Evaluative Judgement 


Evaluative judgement is the ‘capability to make decisions about the 
quality of work of self and others’ (Tai et al. 2018, 5) and not just to make 
those judgements but to defend them, argue persuasively, listen to other 
arguments and consider whether a judgement needs to be modified. 
Evaluative judgement can be developed through peer learning, for 
example through self and peer assessment and through analysing 
exemplars of assignments with others (Boud et al. 2018). Typically, in 
higher education, students may not see examples of their peers’ work and 
this makes it difficult for them to get a sense of the academic standards 
on the programme and to benchmark their own work. Dawson et al. 
(2018) make the distinction between epistemological capabilities (what 
students know and can do) and ontological capabilities (what they are 
learning to be) and argue that evaluative judgement is ‘an epistemic 
capability, which students use in determining if they can act in particular 
situations that are changing dynamically’ (2018, 2). 

Evaluative judgement is not a new concept but using evaluative 
judgement as an organising principle for designing learning and 
assessment is novel (Ajjawi et al. 2018, 7-17). Rather than positioning 
students as passive recipients in assessment practices, evaluative 
judgement places the student at the centre, involving them in assessment 
design and assessment judgements. In order for students to take on this 
role they need to have a good understanding of the academic standards 
in the programme and of what constitutes good quality work (Ajjawi 
et al. 2018). Research in academic literacies shows that notions of quality 
in student work are discipline specific and epistemologically bound (Lea 
and Street 1998), so it is important that students develop an under- 
standing of quality in their discipline(s). Evaluative judgement fits with 
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the notion of sustainable assessment, as students have the opportunity 
to develop a ‘capability that extends beyond the individual and any 
immediate piece of work’ (Ajjawi et al. 2018, 9). Ajjawi et al. (2018, 11) 
suggest five steps for developing informed judgement: 


Identifying oneself as an active learner. 

Identifying one’s level of knowledge and the gaps in this. 
Practising testing and judging. 

Developing these skills over time. 

Embodying reflexivity and commitment. 


od iS 


To develop evaluative judgement, students need opportunities to make 
judgements and calibrate those judgements with others, listening to 
and evaluating information from others so that they achieve ‘an internal 
calibration of quality’ (Ajjawi et al. 2018, 11). The external information 
used to develop calibration judgements comes from peers and teachers, 
for example, ‘feedback comments [from teachers] should be deployed 
to help students calibrate their own judgements ...’ (Boud et al. 2015, 
14). This requires a reconceptualisation of feedback in higher education, 
a reconceptualisation that places students at the centre of feedback 
processes as active participants. Molloy and Boud (2013) propose a 
conception of feedback in which students have control and agency to 
initiate feedback. They distinguish two types of feedback practices which 
they term mark one and two. In mark one, teachers are engaged in 
telling students how to develop. They monitor student achievement and 
adjust feedback on subsequent tasks to guide student development. In 
contrast, in feedback mark two, learners are active and have ‘agency and 
choice’ (Molloy and Boud 2013, 22). Students solicit feedback and use it 
to develop their learning; feedback then becomes a part of sustainable 
assessment as students use it to develop their learning both within the 
university and in professional settings when they leave higher education. 
Solicited feedback from peers and teachers has a role to play in 
developing evaluative judgement; this ‘external’ feedback helps students 
to moderate and develop their own judgement as they consider and 
analyse the evaluations of others. In this way, solicited feedback builds 
‘student capacity for making judgements about their subsequent work’ 
(Molloy and Boud 2013, 22). The external feedback giver challenges and 
helps calibrate the learner’s judgement, encouraging greater objectivity 
on their work. However, it can be demanding for learners to initiate the 
feedback process so building in support for learners in initiating feedback 
is important. Similarly, learners need guidance on what to do with the 
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feedback they receive and how to use it to progress. This has implica- 
tions for programme design as the programme needs to include practice 
tasks that allow students to develop skills and knowledge, and to practise 
receiving and acting upon feedback multiple times. 

Molloy and Boud identify seven features of feedback mark two, 
namely: 


1. Students are orientated not only to standards of work (learning 
outcomes) but also to the purpose of feedback. With this 
explicit orientation, students are more likely to see feedback as 
a process they can use, rather than a tool imposed on them. 

2. Students judge their own work and are encouraged to articulate 
this judgement (self-evaluation). 

3. Students seek or solicit feedback on those aspects of their work 
that matter to them most (for example asking the external 
source to comment on particular aspects of their performance 
that require improvement). This serves to cue educators and 
external providers of information into what to focus on to best 
help learners achieve their goals. This honesty in acknow- 
ledging limitations in their own practice does leave them 
vulnerable, and this honesty can be compromised if students 
are overly attuned to the summative assessment process, that 
is, they are always attempting to ‘show their best selves’ to the 
educator. 

4. Educators or ‘others’ provide performance information to the 
learner. 

5. The learner then engages in a comparative process where they 
combine the internally and externally generated judgements 
and decide how to meaningfully interpret these messages. 

6. The comparison of judgements, and how these relate to the 
standards or goals of work, are used to generate a plan for 
improved work. 

7. The strategies are implemented in the subsequent participa- 
tion in later tasks. 


(quoted from Molloy and Boud 2013, 24) 


Teachers may have different conceptions of quality and may give different 
feedback to learners (see Chapter 6 on marker reliability) so the aim is to 
have dialogue around feedback where student and teacher can negotiate 
understandings of standards. It is unlikely that teachers will have 
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calibrated standards but if they do, they can share calibration resources 
with students and use guided marking to develop student understanding 
of academic standards. 


How to Develop Good Feedback Practices in a 
Teaching Team 


In order to develop student understanding of feedback and assessment 
standards, it is necessary to first develop teachers’ understanding of 
standards. Chapter 6 discusses ways of working towards a consensual 
understanding of standards in a programme team, and in a discipline 
through national calibration. Ensuring discussions across the team is 
key to enabling the team to refer to common academic standards and 
give consistent, good quality feedback to students. This can be achieved 
through peer review of feedback using a tool suchas the feedback profiling 
tool described in Hughes et al. (2015). In a review of the literature, 
they identified five ‘empirically and theoretically derived categories of 
feedback [namely]: praise, ipsative (comments on progress, see below), 
critique, advice and questioning’ (Hughes et al. 2015, 1080). Their study 
drew on Orsmond and Merry’s (2011) categorisations of feedback. 
They found that ‘praise’ was the most frequently used category, while 
‘suggestions for future assignments’ was rarely used. Applying the tool 
to feedback in five modularised postgraduate programmes, they found 
that, in these programmes, feedback from one module did not ‘open up a 
dialogue to continue from one module to the next within a programme’ 
(Hughes et al. 2015, 1080). 

The purpose of feedback is to enable students to effect change in 
the quality of their work: ‘Feedback is a process whereby learners obtain 
information about their work in order to appreciate the similarities and 
differences between the appropriate standards for any given work, and 
the qualities of the work itself, in order to generate improved work’ 
(Molloy and Boud 2013, 6). 

So, if students are unable to effect change, then feedback is 
ineffective. Telling (Sadler 2010) students what to do is transmissive 
teaching, and telling students how to develop, is often ineffective because 
students do not understand the feedback or do not know what to do 
with the feedback. In a teacher-centric approach to assessment, such as 
in feedback mark one, ‘assumptions are made that students will readily 
learn from corrective feedback and prescriptive guidance’ (Hughes et al. 
2015, 1081). For students to make use of feedback, they need to develop 
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constructions of academic standards on their programme, benchmark 
their own work, initiate feedback from others and engage in dialogue on 
how to move their work from where it is now to the next level. 


Reviewing Teachers’ Feedback in a Programme 


Consistency in feedback is important. Students may receive extensive 
feedback from one teacher and significantly less from another. Teachers 
may be choosing to comment on very different issues, so it is important to 
have dialogue in a programme team about what constitutes good quality 
feedback, what areas of feedback should be highlighted and how best 
to give students developmental guidance. One way of ensuring greater 
consistency is to carry out peer review of feedback. Begin by identifying 
an area that needs attention and the level of work you want to start 
with. There may be issues around writing the discussion section of a 
report, carrying out a literature review, student presentation skills and 
developing cogent arguments. Select some student assignments of the 
appropriate level (e.g. first year or final year) which illustrate the issues 
and, with permission, collect teacher feedback on these assignments. 
Anonymise and distribute the assignments and feed back to the teaching 
team. Ask them to read and analyse the feedback given. You might 
suggest that they use the feedback profiling tool (see below) to help with 
this. Use the categories in the tool to categorise feedback comments, e.g. 


So, for example, in the feedback, ‘When you mention Y’s model 
you do so uncritically, but you have responded to feedback on your 
draft and now mention the limitations of applying model Z to your 
design’, the first section is a critique while the second part of the 
sentence is a reference to making progress. This would therefore 
be scored as 1 point for C2 and 1 point for P2. (Hughes et al. 2015, 
1084) 


Continue analysing the feedback comments and count each instance of 
each category, e.g. instances of praise and instances of giving advice. 
Tally the score for each category (the score is the number of instances of 
each category). Rank the scores so that you can easily see a profile of the 
feedback given. 
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Table 8.1: Feedback profiling tool 


Assessment Careers JISC funded 


Assessment Careers Feedback Analysis Tool 


Programme ... 
MO dUle s...cisnassicnsvusisuskpednevednaneds s Gisloasocivnsdeonate ents Student NaMe saci, scisescsedecpectevecneconnsdivns ce innii iadi (to be anonymised) 
Grade if known (to be anonymised) Formative or summative assessment ...........:::ceeeee 
Rank 
Category of feedback Code | Sub-category Examples Score order 
Giving praise P1 ‘A well-constructed argument...’. 
Recognising progress P2 ‘This represents a significant improvement ...’ “You have 
(ipsative) taken on board critique ...’ 
G1 Correction of errors Spelling, grammar, referencing etc. 
C2 Factual critiques (of content ‘Ido not think you can say X. ‘This is not in enough depth.’ 
Critical feedback Gaal F q ( h Gt = y y 8 P 
ritique of approach (structure 
C3 q pp ‘It would have been better to conclude with Y ...’ 
and argument) 
Al Specific (to current assignment) | ‘You might want to consider X ...’ 
General points (specific to e.g. on depth, argument and structure: ‘There is scope to 
Giving advice A2 P . (sp 8 P 8 : ; P 
current assignment) tease out further detail on X ... 
A3 For future assignments ‘In your next essay you should consider Y ...’ 
Clarification requests Q ‘What do you mean by Z?’ 
pn Statements which do not make a judgement, e.g. 
Unclassified statements |O ie 
descriptions of the work. 


Find out more about the feedback profiling tool here: http://jiscdesignstudio. pbworks.com/w/page/50671006/Assessment%20Careers%20Project 


Table 8.1 shows the feedback profiling tool, which you can find out more 
about here: http://jiscdesignstudio.pbworks.com/w/page/50671006/ 
Assessment%20Careers%20Project 

Arrange for teachers to share their thoughts on the feedback, they 
can do this either at a workshop (scheduled alongside a departmental 
meeting, for example, to maximise availability of teachers) or online 
using a virtual learning environment where teachers can anonymously 
post comments on an online forum. Compare scores and profiles across 
the team and discuss disagreements. The aim is to achieve an agreed 
understanding of how to best give feedback and guide students to achieve 
the academic standards on the programme. The feedback profiling tool 
enables teaching teams to reflect on the categories of feedback they are 
giving, how consistent the feedback profile is between teachers and 
whether students are being challenged (through questioning) to develop 
their constructions of good quality work. Watch this video of how one 
teaching team approached peer review of feedback and the guidelines 
they developed for teachers and students: https://mediacentral.ucl. 
ac.uk/Play/8045. 

The feedback profiling tool can also be used with students to help 
develop their understanding of both academic standards and feedback. 
Following a feedback mark two approach, students can initiate feedback 
and analyse feedback using the tool; discussing their analysis with 
teachers could be illuminating for both parties. The categorisation in the 
tool indicates the purpose of feedback comments and facilitates analysis 
of these comments. The tool could also be used with peer assessors (see 
Chapter 7) to analyse and reflect on the feedback they give and receive. 


Ipsative Assessment 


As a result of modularisation, feedback seems to be contained within 
the module, rather than supporting students’ long-term development. 
Hughes has suggested that ‘ipsative feedback’ best supports development: 
‘An ipsative formative assessment activity allows a learner to demonstrate 
progress and change through repeating activities or through comparing 
activities that address comparable knowledges and skills. Ipsative 
feedback then informs the learner of how s/he has progressed, or not, 
since a previous assessment’ (Hughes 2014, 75). 

The competitive nature of modern higher education benefits high 
achieving learners. Ipsative assessment is about ‘being as good as you 
can be at that point in time’ (Hughes 2014, 5) and recognising student’s 
progress not in improved grades but in relation to their previous work. 
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Ipsative assessment is self-referential and not measured against external 
standards. It is like a personal best in athletics; an athlete may not win 
the race, but they may have produced their best performance. Ipsative 
assessment allows assessors (self, peers and teachers) to recognise 
and celebrate progress, even if progress does not result in an increased 
degree classification. 


Using Audio Feedback 


Technology is increasingly playing a role in delivering feedback and 
enhancing the immediacy of feedback for example, audio feedback 
can be given on students’ work and most virtual learning environments 
contain tools for recording feedback (Lunt and Curran 2010). Audio 
feedback is claimed to provide a richer, more personalised experience 
to students as they respond to teacher’s intonation in recordings. A 
recent large-scale study by Zimbardi et al. (2017) tracked undergradu- 
ate student use of both audio and written feedback in a virtual learning 
environment in two biomedical science courses in a research-intensive 
Australian university. Students were required to undertake assessments, 
involving writing reports, to develop their ability to write a scientific 
journal article. The cohorts were large; a total of 5,960 reports were 
submitted and marked by 38 markers who used both typed and audio 
feedback. They found that markers’ audio annotations had nearly eight 
times the number of words as typed annotations, so significantly more 
feedback was given to students in audio annotations. However, the 
quality of the audio feedback was not investigated; the increased word 
count may arise from the kind of padding naturally used in speech. 
Students use of feedback could be tracked through the number of clicks 
and matched to their academic performance. They found high rates of 
opening and use of feedback in both first- and second-year students; 
‘92% of first year and 85% of second-year students accessed their 
feedback, with 58% accessing their feedback for over an hour’ (Zimbardi 
et al. 2017, 625). Both years were less likely to use the feedback from 
the final report, suggesting that students used feedback from the earlier 
report to improve their final report. 
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Teaching idea — Paraphrasing feedback 


To check on student understanding of feedback, ask students to 
paraphrase and explain feedback comments. 

After you or students (peer assessors) have marked and given 
written or audio feedback on an assignment, ask students to look 
over the feedback and either post online a comment that they did 
not understand or a comment they could understand and found 
useful (explaining why it was useful). Ask students to paraphrase the 
comment and explain what they think it means and how they will use 
this comment to develop their work. Then ask them to look at their 
peers’ comments and, if they can, paraphrase any comments their 
peers have not understood. 

Review the comments and identify feedback that students 
appear to have understood and the feedback that students have not 
understood. Use exemplars of students’ work to illustrate feedback 
that students have misunderstood. Repeat the exercise for the next 
assignment to check whether students better understand and can give 
and use feedback. 


Nicol (2010 and Nicol et al. 2014) argues that composing feedback is 
more cognitively demanding than receiving feedback. When students 
articulate what they value in a peer’s work, they are articulating their 
understanding of academic standards on the programme. Students may 
learn more from the process of judging others’ work than from receiving 
feedback comments on their own work. The case study below explores 
what one peer assessor learned from making judgements about students’ 
work and composing peer feedback. 


Case Study: a Peer Assessor Learning from Composing Peer 
Feedback 


Type of programme — BEng 

Mode - blended 

Level — undergraduate 

Cohort — home and international 

What’s of interest — study of what a peer assessor learns from composing 
feedback 
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This case study (McConlogue 2015) describes the experiences of a 
peer assessor, focusing on what she learned from composing peer 
feedback. Ferdous was a second year undergraduate student in medical 
engineering, at the time of the study. She was studying at a UK research- 
intensive university, based in an ethnically diverse community. Ferdous 
was born in Somalia and educated in the UK from the age of 12. She is 
multilingual, describing English as her third language. She was a diligent 
student with previous experience of marking student work at secondary 
school, under the supervision of her teacher. 

In the study, Ferdous took part in peer assessment of a lab 
report. After submitting her report, she attended a ‘rehearsal marking’ 
session (Falchikov 2004). She had been given three lab reports and the 
assessment criteria, and had read and written grades and comments 
on the reports. In the rehearsal marking session, she discussed these 
comments in a group, compared grades and discussed the assessment 
criteria. Each group tried to reach a consensus on the exemplars and the 
teacher collated the results. The teacher joined in the discussion and 
explained how she had assessed the lab reports. She prepared students 
for peer assessment of their lab reports by answering questions about 
technical issues. After the rehearsal marking, students were allocated 
four reports and given two weeks to mark and submit their grades and 
comments. To ensure students saw a range of reports, the teacher had 
pre-sorted the reports into four groups, depending on her judgement of 
their quality, and students were allocated a report from each group; so 
they received a borderline report, some good reports and an excellent 
report (according to the teacher’s judgement). Each report had multiple 
peer markers and marks and comments were uploaded and distributed 
online. 

Ferdous set about peer assessing almost immediately, skimming 
the reports and making notes on the basics, e.g. word length, relevant 
sections etc. She then used her knowledge of the assessment criteria, the 
discussion in the rehearsal marking and the task instructions to begin to 
write more detailed comments on the reports. She diligently composed 
around 500 words of peer feedback for each report and estimated that it 
took her about four to five hours to mark the four reports. 

She seemed to learn a great deal from reading and commenting 
on the reports. She benchmarked her own work against her peers. She 
learned from the weaker reports, seeing things that had been done well 
in them. She said she could see ‘negatives and positives in everyone’s 
report’. Seeing the range seemed to help her develop ideas of how to 
improve her own work and what she needed to do to move on to the next 
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level. The process of comparing her work and her peers’ reports, writing 
comments and interpreting the assessment criteria seemed to develop 
her understanding of academic standards. Interestingly, seeing a range 
of reports seemed far more important than seeing an ‘excellent’ report. 
Ferdous commented that the excellent report was ‘complicated’ and that 
she ‘couldn’t replicate it’, suggesting that it was too far removed from her 
own stage of development. The other reports helped her develop ideas 
and strategies for moving on. 


Further Reading 


Falchikov, Nancy. ‘Involving Students in Assessment’, Psychology Learning 
and Teaching 3, no. 2 (2004): 102-8. 

McConlogue, Teresa. ‘Making Judgements: Investigating the 
Process of Composing and Receiving Peer Feedback’, Studies in Higher 
Education 40, no. 9 (2015): 1495-506. 

Nicol, David, Avril Thomson and _ Caroline Breslin. 
‘Rethinking Feedback Practices in Higher Education: A Peer Review 
Perspective’, Assessment and Evaluation in Higher Education 39, no. 1 
(2014): 102-22. 


Follow-Up 


Reading 


On evaluative judgement: 


Tai, Joanna, Rola Ajjawi, David Boud, Phillip Dawson and Ernesto 
Panadero. ‘Developing Evaluative Judgement: Enabling Students 
to Make Decisions about the Quality of Work’, Higher Education 76, 
no. 3 (2018): 467-81. 

Boud, David, Rola Ajjawi, Phillip Dawson and Joanna 
Tai. Developing Evaluative Judgement in Higher Education: Assessment 
for Knowing and Producing Quality Work. London: Routledge, 2018. 


On what students learn from composing feedback: 
McConlogue, Teresa. ‘Making Judgements: Investigating the 


Process of Composing and Receiving Peer Feedback’, Studies in 
Higher Education 40, no. 9 (2015): 1495-506. 
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Nicol, David, Avril Thomson and Caroline Breslin. 
‘Rethinking Feedback Practices in Higher Education: A Peer Review 
Perspective’, Assessment and Evaluation in Higher Education 39, 
no. 1 (2014): 102-22. 


Think about how you could provide opportunities to develop students’ 
evaluative judgement on your programme. What activities could you 
introduce? For ideas see: 


Boud, David, Rola Ajjawi, Phillip Dawson and Joanna Tai. Developing 
Evaluative Judgement in Higher Education: Assessment for Knowing 
and Producing Quality Work. London: Routledge, 2018. 


Investigating Your Practice 


Investigate feedback in your team. Carry out a peer review of feedback 
(see a suggested plan in the section ‘How to develop good feedback 
practices in a teaching team’ above). Use the feedback profiling tool or 
similar to analyse feedback. Involve the team and students in creating 
guidelines for feedback. Implement the new guidelines and follow-up 
with an evaluation of teacher and student perceptions of whether the 
quality and usefulness of feedback improved. 
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Part 3 
INCLUSIVITY 


Chapter 9 


Developing Inclusive Curriculum and 
Assessment Practices 


The widening participation agenda in higher education has brought more 
diverse groups of learners into higher education; these learners often 
have non-traditional educational backgrounds, and this necessitates a 
change in the curriculum, including assessment, to meet the needs of a 
diverse student body. This chapter explores diverse learner character- 
istics, such as social background, disability, gender, race and sexuality, 
and how these characteristics intersect to create multiple exclusions. The 
chapter discusses how assessment design needs to change in order to 
be more inclusive and practical solutions, such as choice in assessment, 
are proposed. Good, inclusive assessment practices can help all students 
achieve their potential. 

‘Inclusive assessment does not compromise academic or profes- 
sional standards but improves the opportunities for all students to 
demonstrate their acquisition of the learning outcomes’ (University of 
Plymouth, 2016). 

You have probably seen the ‘fair test’ cartoon. A bird, monkey, 
penguin, elephant, fish, seal and dog are all given the same test and 
asked to climb a tree. For some this is an immensely easy task, for 
others impossible. Similarly, with diverse student cohorts in higher 
education, setting all students the same task does not ensure fairness. 
Field trips that exclude students with mobility issues or students with 
caring responsibilities are not equitable. Types of assessment can also 
discriminate. Lillis (2001) critiques the traditional academic essay as 
the default assessment task which, she claims, advantages students 
from certain educational backgrounds while disadvantaging students 
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from non-traditional educational backgrounds or students with learning 
differences such as dyslexia. Developing fair and equitable assessment 
and programmes of study entails ensuring that diversity is represented 
in the curriculum and in types of assessment, and that students are fully 
involved in assessment design. Choice in well-designed assessments 
enables students to demonstrate their learning and avoids disadvantag- 
ing groups of students. 

Working with students to explore inclusive curriculum design can 
be immensely rewarding for both teachers and students; challenging 
traditional curricula can stimulate new thinking within the discipline. 
Inclusive curriculum design involves not only changing content but 
also the way we teach and assess; programmes need to incorporate 
fair and equitable assessment. Anti-discriminatory legislation identifies 
protected characteristics such as age, disability, gender, race and 
sexuality. Scholarly work, such as work in the fields of disability studies, 
gender studies, critical race awareness and queer studies, has raised 
awareness of the discriminatory nature of traditional curricula and 
assessment practices (Moore-Cherry et al. 2016). Much work is being 
done worldwide to challenge curricula and research paradigms that 
marginalise others. 

In a seminal article, Kimberlé Crenshaw (1989) identified ways 
in which individuals belonging to several marginalised groups were 
multiply disadvantaged and called this ‘intersectionality’. She explored 
the concept of ‘intersectionality’, highlighting ways in which multiple 
interlocking characteristics combine to create multiple disadvan- 
tages. Crenshaw proposed an influential analysis of systems of power 
within societies which create structures and processes that benefit the 
dominant group and work to exclude marginalised groups. Marginalised 
groups are excluded from systems of power in society; individuals who 
intersect several marginal groups are multiply excluded. For example, 
an individual with a disability, belonging to an ethnic minority and 
identifying as non-binary may face multiple disadvantages. All these 
characteristics, each of which may individually trigger discrimination, 
can interact to multiply exclude. 


Barriers to Participation 
Academia has traditionally been populated and led by middle-class 


white men. This demographic is changing but the influence of this legacy 
may linger on in curricula, and affect who teaches, what is taught, how 
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teaching is done and how students are assessed. Traditional ways of 
thinking and carrying out research in the disciplines may exacerbate this 
exclusion. Henrich et al. (2010) argue that behavioural scientists use 
an unrepresentative population sample in research studies. They claim 
most studies use ‘Western, educated, industrialized, rich and democratic 
(WEIRD)’ (Heinrich et al. 2010, 29) participants, typically American 
undergraduates. Results are then skewed because of the restricted nature 
of the sample and yet are often generalised to the human population as a 
whole. They argue that journal editors and grant awarding bodies need 
to work with researchers to ensure that diverse populations are studied 
in order to ‘develop interdisciplinary, international research networks 
for long-term studies on diverse populations using an array of methods, 
from experimental techniques and ethnography to brain-imaging and 
biomarkers’ (Heinrich et al. 2010, 29). 

Research in developing countries is often not published in 
mainstream, English-medium international journals. Websites that 
promote research from non-Western countries are appearing and 
provide dissemination of scholars’ work (see for example the African 
Journals Online library which hosts 523 journals, including 260 
open-access journals, https://www.ajol.info). Another barrier to 
research is the considerable cost of equipment and laboratories. Burns 
et al. (2019) report on a neuroscience study that overcame this problem 
by using portable equipment to study persuasion in Jordan with an Arab- 
speaking sample. While Muthukrishna et al. (2018), in an effort to make 
the literature less WEIRD, produced a cultural distance scale to identify 
societies that could usefully be compared. The authors explain the scale: 


[...] provides a means to measure the psychological and cultural 
distance between two societies and create a distance scale with any 
population as the point of comparison. Since psychological data is 
dominated by samples drawn from the United States or other WEIRD 
nations, this tool provides a ‘WEIRD scale’ to assist researchers in 
systematically extending the existing database of psychological 
phenomena to more diverse and globally representative samples. 
(Muthukrishna et al. 2019, 2) 


They claim that ‘... there exists no systematic method for determining 
which societies will provide useful comparisons or even the size of the 
psychological differences—the cultural distance—between societies, be 
they non-Western, less-educated, less-industrialized, poorer, non-demo- 
cratic or some subset of these’ (Muthukrishna et al. 2019, 1) and that the 
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cultural distance scale fills this gap. The cultural distance scale can be 
viewed here: http://culturaldistance.muth.io. 


Social Background 


Research on social background and access to education shows inequality 
in participation in higher education. Students from low-income 
households are less likely to study at universities, especially elite univer- 
sities. This seems to be the picture worldwide; in a thorough report 
on government policy commitments to equity in higher education 
worldwide, Salmi reports that: 


[a]vailable data show that participation in higher education 
continues to be unequal from a social background perspective. 
Research produced by UNESCO in 2016, looking across 76 mainly 
low-income countries, found that only 1 per cent of the poorest 
25-29-year olds had completed at least four years of higher 
education, compared to 20 per cent of the richest. Furthermore, 
disparities in access to higher education are amplified by 
inequalities in success by social background during the course of 
studies. (Salmi 2018, 5) 


In addition to social background, other characteristics intersect to 
exclude students, for example, in China, Li (2015) found a participation 
gap between urban and rural poor students. 

Research on admissions to elite universities, indicates that social 
background inequity is more pronounced, with lower rates of participa- 
tion from students from low-income households. 

Reasons for this inequity may vary from country to country. In the 
UK, Anders (2012) investigated the link between household income and 
university entry and concluded that universities do not discriminate, but 
that poorer students are less likely to apply to elite universities. Anders 
claims: 


These findings suggest that policies aimed at reducing the 
university participation gap at the point of entry are likely to face 
small rewards. More likely to be successful are policies aimed at 
closing the substantial applications gap, particularly by ensuring 
that students from poorer backgrounds have the necessary 
qualifications to apply. (2012, 185-6) 
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In China, Luo et al. (2018) found that changes in entrance qualifica- 
tions advantaged students from richer households applying to elite 
universities. 

Government policies designed to increase equity tend to focus 
on financial aid for poorer students (Salmi 2018). The most advanced 
countries ‘have a comprehensive equity strategy to promote equality of 
access to higher education; these are typically rich countries, but Cuba 
stands out as a country that has promoted equity since 1959 (Salmi 
2018, 11). 

Financial incentives are important but Salmi also argues for 
interventions to increase attainment and success. Belonging interven- 
tions that affirm students’ values and stress that different backgrounds 
matter, have been shown to be successful in reducing the social-class— 
achievement gap in the USA (Stephens et al. 2014; Harackiewicz et al. 
2014). Stephens et al. (2014) found that their intervention, valuing 
diverse student backgrounds, increased first generation student grades 
and increased their take up of university support services. 


Black and Minority Ethnic (BME) Attainment Gap 


Researchers in the UK have identified a gap in attainment between 
white students and black and minority ethnic (BME) students (Woolf 
et al. 2011; Richardson 2008). In a meta-analysis of 23 reports on the 
performance of medical students, Woolf et al. (2011) found that ethnic 
differences were widespread and significant, that the attainment gap has 
existed for over three decades and is found in both undergraduate and 
postgraduate assessment. Mountford-Zimdars et al. (2017), in a wide- 
ranging report on the attainment gap, suggest that institution-wide 
interventions that address both the curriculum and assessment design 
are most likely to effect change (see Mountford-Zimdars et al. (2017) for 
a range of suggestions). 

Decolonising the curriculum movements worldwide call for a 
critical scrutiny of curricula that has historically favoured Western, white, 
male, middle-class perspectives; these perspectives are often reflected in 
curriculum content and reading lists. There is a move to review reading 
lists with students to identify cultural bias and include more texts and 
perspectives from marginalised scholars. Controversial areas of research, 
for example eugenics, are being investigated and re-evaluated (see UCL’s 
inquiry into eugenics and Francis Galton’s legacy, UCL 2018). 

In medical education, the needs of marginalised groups can often 
be overlooked. Gishen and Lokugamage (2019) report on a project to 
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diversify and decolonialise a medical education programme. Working 
with teachers, students and the public they have changed the medical 
curriculum and raised awareness of the health needs of marginalised 
groups. For example, medical students are taught to recognise signs 
of anaemia and cyanosis in BME patients. Case studies from a range of 
marginalised groups are used to raise awareness of issues such as caring 
for a transgender male patient with breast cancer. 


Disability and the Social Model 


Disability scholars have described two contrasting models of disability 
(Oliver 1995, 2004). The first, and the traditional model, they have 
termed the ‘medical model’ of disability. The medical model focuses on 
impairments and ways of ‘fixing’ them, essentially fixing the individual 
so that they can function and fit in with the current societal norms. In 
contrast, the social model argues that it is the way society is organised 
that marginalises and restricts people with disabilities, putting up 
barriers to their full participation in society. Shakespeare explains 
the social model makes ‘... the distinction between disability (social 
exclusion) and impairment (physical limitation)’ (2006, 215). 

The issue becomes how to structure society so that it is inclusive 
and accepting, valuing difference, instead of creating barriers to partici- 
pation. Removing barriers to participation in educational activities gives 
disabled students access to learning opportunities on their programmes 
of study. In the social model, disabilities are not seen as impairments but 
as ‘human variations’ (Felten and Bauman 2013, 370); a diverse student 
cohort with a range of human variations brings a rich variety of perspec- 
tives to a field of learning: 

Taking an inclusive approach to partnership often requires staff and 
institutions to reframe their perceptions of students (and colleagues) 
who have traditionally been marginalised. For instance, deafness is 
commonly understood as hearing loss. As a result, deaf people are seen 
as needing to be ‘fixed’ or ‘cured’ before they can be full participants 
in the community. That belief, however, is often at odds with the life 
experiences of many deaf people. Instead of considering deafness as a 
loss, it also can be recognised as: 


‘an expression of human variation that results in bringing to the 
fore specific cognitive, creative, and cultural gains that have been 
overlooked within a hearing-centered orientation’ (Felten and 
Bauman 2013, 370). In this conception, hearing loss gives way to 
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‘deaf-gain’ (Bauman and Murray 2010). Rather than focusing on 
real or perceived deficits of certain groups of students, adopting 
a ‘deaf-gain’ perspective highlights the distinct capacities, assets 
and valuable perspectives that different students bring to the 
co-creation of learning and teaching through, for example, sharing 
of classroom experiences from a range of perspectives to enable 
thoughtful pedagogical redesign for the benefit of all students and 
staff. (Bovill et al. 2016, 204) 


Creating an Inclusive Curriculum and Designing Inclusive 
Assessment 


Collaborating with Students 


An inclusive curriculum and inclusive assessment practices can be created 
by working in partnership with students and community members. 
Changes in medical curricula have been devised by involving patients in 
redesign; helping medical students to better understand the perspectives 
of marginalised groups and how the curriculum and clinical practice 
can be shaped by these perspectives. For example, in ophthalmology, 
Rosie Gilbert describes how she invited sight-impaired patients into her 
course to give students the patient perspective of eye disease. A student 
commented that in labs she works with a protein or genes and does not 
connect this work with patients. By inviting patients into the course, 
Rosie feels students get a better understanding of the emotional aspect of 
visual loss. You can see a video of this work here: https://www.youtube. 
com/watch?v= gGjeBK4PqmM&feature=youtu.be&yt%3Acc=on. 
Similarly, addressing the needs of LGBT+ patients with medical 
students raised awareness of LGBT+ health needs. Jess Salkind, a 
junior doctor, and Ginger Drage describe involving LGBT+ patients in 
a workshop to educate medical students about LGBT + health issues to 
overcome the lack of training in this area. The workshop was a compulsory 
part of the medical programme for fifth year (penultimate year) medical 
students and covered the health inequalities experienced by LGBT+ 
patients. There were discussions of scenarios and good practice. Most 
importantly, trans visitors spoke about their experiences of health 
care in a supportive environment and encouraged medical students 
to ask whatever questions they wanted. The workshop was evaluated, 
and participants were asked how comfortable they felt taking medical 
history from a trans patient. Before the workshop 26 per cent said they 
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felt comfortable, after the workshop 82 per cent felt comfortable. You 
can watch a video about the workshop here: https://www.youtube.com/ 
watch?v=kBbawKTfaEE&feature =youtu.be&yt%3Acc=on. 

Read more about trans patients and health in an article written by 
transgender authors (Lewis et al. 2017) in the British Medical Journal 
(BMJ). 


Inclusive Assessment: Practical Implications 


An inclusive curriculum necessitates inclusive assessment. Inclusive 
assessment involves removing barriers to learning and enabling 
students to demonstrate knowledge and skills in a form that allows 
them to excel. This can be done through a combination of ‘modified 
assessment provisions (MAP)’, and an assessment design that allows 
learners to choose a type of assessment that showcases their learning. 
O’Neil explains: ‘Inclusive assessment benefits students as they can chose 
assessment to suit their strengths, learning style, needs, time constraints, 
personal or employment commitments’ (2011, 14). 

Providing choice in assessment is not only good for learners, it is 
also more efficient for teaching and administrative staff. Allowing choice 
reduces the need for complicated modified assessment provisions which 
can be costly, e.g. providing more time in examinations necessitates 
funding additional invigilators (O’Neill 2013). Inclusive assessment 
design is essentially good practice. Students benefit from being involved 
in designing assessment tasks, constructing assessment criteria and 
making judgements about their own and their peers’ work (see Chapter 7 
for guidance on peer assessment). Through this involvement they are 
helped to gain a good understanding of assessment standards and helped 
to monitor and develop their work. 

Introductory modules or formative modules which help students 
to understand ways of thinking in the discipline (Entwistle 2005) and 
also assessment method(s), promote equity in diverse cohorts. These 
modules typically run at the start of a programme and give students 
practice in understanding and analysing key ideas, critical appraisal 
and practice in ways of communicating ideas within the disciplines in 
oral and written assignments. Thinking Writing (2019) activities, such 
as short write-to-learn tasks and longer writing tasks, embedded in 
the discipline teach students how to communicate thinking in written 
texts and support writing development. This is particularly important 
for students from non-traditional backgrounds. Interventions aimed at 
supporting student thinking and writing can be found in most disciplines, 
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e.g. psychology (Wentworth et al. 2017). Divan et al. (2015) found that 
most international students on a UK master’s programme in biology 
agreed that writing assignments on the course were different from their 
previous studies. Divan et al. (2015) introduced a writing intervention at 
the beginning of the course and compulsory sessions in academic writing 
in the first two weeks; they claim this intervention reduced plagiarism. 


Designing Choice in Assessment 


Choice in assessment does not mean giving an advantage to a group of 
students. Equity in assessment choice is key, so it is important to create 
a range of assessments of similar complexity and involving similar effort 
and time from students. Some students prefer to take examinations; they 
have developed strategies to enable them to cope with the pressurised 
workload and stress of exams. Others find this pressure debilitating and 
are prevented from doing their best work in an exam. For these students, 
an alternative assessment method is needed. Exam anxiety seems to 
affect students from a diverse range of cultures; Arana and Furlan (2016) 
investigated exam anxiety in Argentine students and found that perfec- 
tionists were more likely to suffer. The method of assessment was found 
to affect test anxiety in Saudi undergraduate medical students with 
students rating long case examinations and objective structured clinical 
examinations (OSCEs) as the most stressful (Guraya et al. 2018). 

The Universal Design for Learning (UDL) project addresses the issue 
of engaging all learners through a range of learning design using diverse 
media and inclusive assessment practices. Comprehensive guidance 
and resources for designing and assessing learning are available on the 
UDL (2019) website. The website provides practical advice on ways of 
designing learning and assessment to address learner variability and 
enable all learners to articulate their knowledge and demonstrate their 
learning. Barriers for learners and ways of removing these barriers from 
assessment tasks are discussed: 


Possible Barriers 
Assessments that have a single response mode (asking learners to 
draw diagrams for every answer) 


Using the same format for response for all assessments (using 
only multiple choice, written response) 
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Assessment Options 


The demands associated with responding to any assessment 
are not always construct relevant. In other words, if the goal of 
the assessment is to understand the processes involved in cell 
division, then asking the learner to write an essay and grading 
the construction of the essay may not be relevant to the task of 
understanding that the learner really needed to know. Supporting 
strategic and organizational abilities and allowing students to 
express acquired knowledge and skills in diverse ways can be built 
into a variety of assessments. 

Choices, again, can be helpful. For example, an assignment 
on identifying barriers to economic expansion in post-World 
War II Europe could include the options of an essay, PowerPoint 
presentation, video, or infographic to achieve the same goal. 
Demonstrating understanding of text can be expressed by students 
choosing keepers (items from the text the learner finds important 
and relevant) and generating queries (questions created by the 
learner to be answered later) about the content. These can be 
expressed in written or verbal form, in a group or alone. 


(UDL, http://udloncampus.cast.org/page/assessment_udl) 


An issue in designing assessment choice is whether assessment tasks are 
equivalent. Attempts have been made to compare types of assessment and 
student effort in order to develop task equivalences. This is problematic 
as equivalences depend not only on word length or effort but also on 
the complexity of the task; a short piece of writing, like an abstract, can 
involve more thinking and redrafting than a longer piece. Some institu- 
tions have attempted comparisons of types of assessment task in order to 
ensure equity. For example, a project at London South Bank University 
established equivalences to a 1,000-word written assignment (University 
of Plymouth, 2016). Assessment type equivalences included a verbal 
presentation of 20 minutes or a group presentation of ten minutes per 
member. Group essays of around 750-1,000 words per member and 
unstructured journal reflections of 2,000-3,000 words were also judged 
to be equivalent to a 1,000-word essay. Roehampton University, ‘What’s it 
Worth’ project has suggested assessment equivalences for a twenty credit 
module, for example, they suggest a 4,000-6,000-word coursework 
assignment is equivalent to a 120 minute written exam or a 60 minute 
practical exam such as a viva (University of Plymouth, 2016). Drawing on 
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this work, teachers could set an assessment task, e.g. researching a key 
concept, and allow students to choose whether to present their work in a 
1,000-word report or a 20-minute presentation. It is obviously difficult to 
make these task comparisons; group work can be more time consuming 
than individual work and frustrating if the group has inefficient working 
practices. Task equivalences may also be affected by discipline. Discuss 
task equivalences with colleagues and students and find a consensus 
that works in your teaching context. You may need to refine task equiva- 
lences, but offering all students choice of task, and guidance on how to 
make that choice, helps to make assessment more inclusive. 

The advantage of assessment choice for students is that they can 
choose an assessment type that plays to their strengths and allows 
them to show what they can do. For teachers and administrators, the 
advantage is that students’ needs are met without complicated and 
costly alternative arrangements, such as additional examination time 
and longer extensions. However, a key principle is that assessment 
choice should not result in easier assessments, or assessments that avoid 
testing key aspects of learning. Similarly, the academic standards for 
the assessments must be equivalent; developing an understanding of 
what teachers value in student work at each level of the programme is 
important. Involving students in creating assessment criteria helps share 
standards. Plymouth University offer the following guidance for thinking 
through assessment choice. 


1. Consider offering a choice of two assessment methods in a 
number of modules, this may reduce the numbers of complex 
modified assessments required. 

2. Students have different strengths, learning styles, time 
commitments; these affect their performance in assessment. 

3. Some simple modified assessment provisions may still be 
required (e.g. the use of computers in written exams and extra 
time in tests). 

4. Ensure both assessment methods use the same detailed 
assessment criteria. 

5. Consider offering all students a choice of typing or handwriting 
in examinations. 

6. Are the assessment methods commensurate and equitable? 
Consider the equity of hours, effort and standards. 

7. Is the assessment method suited to individual, group work or 
both? 
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8. When offering assessment choice what is the ‘default’ 
assessment method? This can be used if students fail to inform 
staff of their assessment choice? (Quoted from Plymouth 
University 2016,14) 


Academic standards must remain the same, irrespective of which method 
of assessment is used. Developing similar criteria to make judgements 
about a range of tasks can help to align standards, but remember that 
criteria are open to interpretation, so activities to discuss and share 
understanding of criteria are essential. Use of exemplars and discussion 
between teachers and students can clarify appropriate standards (see 
Chapter 6). 

Guidance for designing inclusive assessment (adapted from 
Plymouth University 2016, 7). 


1. Ensure students have a good understanding of standards on the 
programme through guided marking activities (see Chapter 6), and 
formative tasks with peer and teacher feedback. 

2. Check assessment tasks to ensure there is no inherent bias that may 
disadvantage particular groups of students. For example, field trips 
may disadvantage students with limited mobility. Study abroad 
opportunities may disadvantage marginalised groups of students 
who do not feel safe in countries where LGBT+, gender and race 
are discriminated against. It is important to involve students in 
planning field trips and study abroad opportunities and ensure that 
appropriate alternatives are provided. If field trips are inaccessible 
or study abroad typically occurs in countries with discriminatory 
practices, consider cutting these activities from the programme and 
developing more inclusive opportunities. Additionally, family and 
caring responsibilities and financial issues may exclude students 
from taking part in any activity organised outside of normal study 
hours and in other locations. 

3. Are all assessments designed to minimise plagiarism and academic 
dishonesty? Consider setting different assessment questions or case 
studies for each student. Ensure that the same assignment/exam 
questions are not used every year. Research-based learning tasks 
that require students to work on different topics (e.g. the mystery 
specimen, see Chapter 5) help minimise plagiarism. 

4. Are assessments designed to allow students to follow and develop 
their own interests with guidance from teachers? 
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5. Check your programme for bunching of assessments. Plan 
assessment across a programme (not just in individual modules, 
see Chapter 4) and across the year. Ensure teachers and students 
have time to give and receive feedback. Schedule time into the 
programme to do this. 

6. Are pre-assessment discussions with students and activities such as 
guided marking organised? Are post-assessment feedback sessions 
organised? 

7. Do markers have a consensual understanding of academic 
standards on the programme? Are calibration activities scheduled 
so markers can agree standards? (See Chapter 6 for guidance.) 

8. Is marking time scheduled into the programme to ensure markers 
can meet feedback deadlines? Avoid scheduling other activities 
that would demand markers’ time. 

9. Ensure assessments are aligned across a programme so that 
feedback from one assessment can be used to improve the next 
assessment. (See Chapter 4 on designing assessment across a 
programme.) 


Inclusive assessment practices benefit all students. Students with 
disclosed disabilities may have a statement of reasonable adjustment 
(SORA) but many students may have undisclosed disabilities with no 
access to reasonable adjustments; by ensuring assessment is inclusive all 
students can choose appropriate methods to demonstrate their learning. 


Follow-Up 


Reading 


On intersectionality read: Crenshaw, Kimberlé. ‘Mapping the Margins: 
Intersectionality, Identity Politics, and Violence Against Women OF 
Colo’, Stan. L. Rev. 43 (1990): 1241. 

On decolonising and disability read: Meekosha, Helen. 
‘Decolonising Disability: Thinking and Acting Globally’, Disability and 
Society 26, no. 6 (2011): 667-82. 

On developing student writing read: Bean, John C. Engaging Ideas: 
The Professor’s Guide to Integrating Writing, Critical Thinking, and Active 
Learning in the Classroom. 2nd ed. San Francisco: Jossey-Bass, 2011. 

On social background and inequality, the National Education 
Opportunities Network (NEON) works to help widen access to higher 
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education. Their website has an excellent range of resources and events: 
https://www.educationopportunities.co.uk. 

NEON sponsors the World Access to Higher Education Day 
(WAHED) website, with resources and research on education access 
worldwide at www.worldaccesshe.com. 


Investigating Your Practice 


Think about how you could introduce choice in assessment in your 
module(s) or programme(s). What challenges would you face and 
how could these be overcome? For ideas, see O’Neill, Geraldine, ed. A 
Practitioner’s Guide to Choice of Assessment Methods within a Module: Case 
Studies for University College Dublin. Dublin: UCD Teaching and Learning, 
2011. Available at: http://eprints.teachingandlearning.ie/3266/1/ 
O’Neil1%202010%20A%20Practitioners%20Guide.pdf. 

Review your curriculum. Working with colleagues and students, 
carry out a review of your curriculum. There are various tools you can 
use: 


The Toolbox Diversity in Education website at Utrecht University 
suggests further reading, gives examples of good practice and 
provides tools that can be used to review curricula: https://xerte. 
uu.nl/play.php?template_id=1127. 

The UCL Inclusive Curriculum Health Check offers a 
short questionnaire for reviewing teaching and assessment on 
your programme: https://www.ucl.ac.uk/teaching-learning/ 
sites/teaching-learning/files/ucl_inclusive_curriculum_ 
healthcheck_2018.pdf. 

Another, longer checklist can be found here, from Edith 
Cowan University: https://intranet.ecu.edu.au/__data/assets/ 
pdf_file/0006/772791/checklist-for-an-inclusive-curriculum.pdf. 
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Chapter 10 
Conclusion 


In this guide, I have aimed to help you bridge the gap between theory 
and research on teaching, learning and assessment and your practice. I 
have aimed to show how theoretical ideas and findings from research can 
be used to help you develop as a teacher and to enhance your assessment 
practices. But I have also emphasised the importance of your practical 
knowledge (Elbaz 2019), the knowledge that you have acquired through 
years of learning and teaching in higher education. Exploring this 
practical knowledge is key to your growth as a teacher. No two teaching 
contexts are the same, so it is unlikely that what works in one context 
can be transferred and used effectively in another without any changes. 
Adaptations are needed to ensure that an approach or activity works well 
in your context. There are ideas for exploring your assessment practices 
at the end of every chapter (investigating your practice) and I hope you 
have found these enjoyable and illuminating. I hope that by working with 
colleagues and students you have gained a more profound understand- 
ing of how assessment can help your students learn, and that together 
you have devised ways of implementing and evaluating new approaches. 

A key concept in this guide is that both teachers and students 
need to work together to develop an agreed understanding of academic 
standards on their programme and that academic standards are socially 
constructed (Sadler 2014; Butler Shay 2004). This involves teachers in 
dialogue with colleagues to explore and harmonise their beliefs about 
disciplinary standards. Students engage in this dialogue and share their 
beliefs, and together this community of teachers and students can come 
to a shared perspective on academic standards, as far as possible. Given 
the nature of learning and performance in higher education, there will 
always be differences in judgement of the quality of assessed work, 
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but by analysing exemplars and calibrating judgements, it is possible 
to come closer to a consensus. Consensual understanding of academic 
standards can then be used to review and design programmes of study; 
when you know what level of work students should reach by the end of 
a programme, it is easier to work backwards through the programme 
and collaborate with students to design scaffolding tasks to help them 
reach that standard. Similarly, a consensual understanding of academic 
standards is crucial to making reliable assessment judgements. With 
this knowledge, teachers can more quickly and more accurately grade 
students’ work. More importantly, students can self- and peer assess and 
become confident, self-directed learners. 

Throughout the guide, especially in Chapter 9, I have emphasised 
that assessment should be inclusive. The decolonising the curriculum 
movement has highlighted the exclusive nature of curricula in higher 
education and the need to review and bring in diverse perspectives 
from previously marginalised groups. Students have taken a lead in 
this movement, agitating for change and asserting their right to bring 
knowledge from their communities and cultures into the curriculum. 
Alongside this movement for change in the curriculum is the impetus 
to open up assessment practices. This ensures inclusivity through 
developing choice and preparing students for appropriate assessment, 
so that they have the means to fully demonstrate their achievements. 
Chapter 9 in this guide suggests ways of achieving this and of reviewing 
your curriculum and assessment practices to ensure inclusivity. 

Marking assessments can seem like a chore or source of stress 
for teachers, but assessment can be a rich learning experience for 
both teachers and students. In Chapter 5, I reported on inspiring work 
where students direct and develop research which contributes to their 
community and knowledge building in their disciplines. This work can 
be enormously enriching for teachers, introducing them to new areas 
of their discipline. Wills (2014) describes how her students engaged 
with local groups to contribute to knowledge making in the community 
and to engage in democratic processes, effecting change. Chang (2005) 
described how he enabled students to research and build knowledge 
in their discipline, publishing their ground-breaking work. He drew on 
students’ work in the exam, cited them as experts and helped students 
develop an inheritance mechanism to hand over their notes and data to 
their successors. In this way, a community of researchers was created, 
building on the work of predecessors. These expert students are 
able, with guidance, to become fully involved in assessment, develop 
evaluative judgement and make competent assessments of their peers’ 
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work. They have the integrity to give good quality feedback to peers 
and learn from this experience. Nicol (2014) argues that composing 
peer feedback is more demanding and a richer learning experience than 
receiving feedback. Peer and collaborative assessment is a tremendously 
valuable learning experience for students that helps them competently 
judge their own work and enables self-directed learning. Collaborative 
assessment enables students to become full members, alongside their 
teachers, in a community of practice, setting academic standards and 
making judgements about the quality of their own and their peers’ work. 
The staged activities described in Chapter 7 demonstrate how you can 
scaffold learning and assessment to help students move from legitimate 
peripheral participation in this assessment community of practice, to full 
members. See Chapter 2 for more discussion of communities of practice 
and legitimate peripheral participation. 

Research-based learning and collaborative assessment can be 
powerful learning experiences for students but, as Brew (2006) points 
out, teachers have more academic capital than students. Teachers can 
strongly influence students, so the trick is to share knowledge and support 
students without dominating or suppressing creativity. Scaffolding 
learning tasks, such as identifying tasks (e.g. Meet the Researcher in 
Chapter 2), can help students explore ways of thinking and practising in 
the discipline without crushing their originality. Scaffolding identifying 
tasks helps students to develop and tackle more open assessments, 
eventually leading to authoring tasks, involving knowledge building. 

If you have read this entire guide, you are obviously a committed 
and motivated teacher. I hope you have been encouraged and stimulated 
by ideas within this guide, and that you will spend some time thinking 
about what you have learnt and how you can apply that to your teaching. 
Has anything struck a chord with you and illuminated an aspect of your 
teaching? What do you want to explore further and how can you engage 
with others, colleagues and students, in this exploration? You may find 
it useful to look back over the investigating your practice sections at the 
end of every chapter and refine ideas for the areas you want to develop. 
Good luck with your journey. 
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Teachers in higher education spend much of their time on assessment, yet many have 
received minimal guidance on assessment design and marking. This means assessment 
can often be a source of stress and frustration. Assessment and Feedback in Higher 
Education aims to solve these problems. Offering a concise overview of assessment 
theory and practice, this guide provides teachers with the help they need. 


In education, theory and practice are often poorly linked. In this guide, Teresa McConlogue 
presents theoretical ideas and research findings and links them to practice. She considers 
recent theoretical work on feedback and suggests ways of developing evaluative 
judgement. Throughout the book, teachers are encouraged to examine their practice 
critically, and there are ideas for small-scale educational investigations, involving 
teachers, their colleagues and students, such as using the Assessment Review 


Questionnaire to adapt assessments. 


A key principle of Assessment and Feedback in Higher Education is that an 
understanding of academic standards is fundamental to good assessment design and 
more reliable marking. The guide explores the concept of academic standards and 
proposes methods of co-constructing shared standards within a teaching team and with 


students through calibration activities. 


Teresa McConlogue is Principal Teaching Fellow in the UCL Arena Centre. She has 
worked co-operatively with higher education teachers and their students in a range of 
contexts and countries. She is interested in how teachers can explore and make sense of 
their practice through small-scale, context-rich investigations, and has investigated ways 
of involving students in assessment judgements and what teachers and students learn 


from collaborative assessment practices. 
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